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THE SECRETARY OF EDUCATION 

WASHINGTON, DC 20002 




September 2004 
Dear Practitioner: 

The rigorous assessment and accountability provisions in the landmark No Child Left Behind Act are 
challenging us as educators to explore new ways to improve instruction in our nation’s schools. One 
powerful tool in this transformation is technology. 

During the last decade, schools were connected to the Internet, and computers were put in classrooms. 
We now must learn how to use this technology more effectively to improve instruction and transform 
education as we know it. 

To generate and share ideas on how technology can assist educators, I convened two technology summits 
to which I invited technology experts and education practitioners. As part of the summits, I 
commissioned a number of white papers on a range of topics. I am pleased to share the compilation of 
these papers with you in Helping Practitioners Meet the Goals of No Child Left Behind. 

This resource is designed to provide school leaders and policymakers with information on ways e- 
leaming can help schools meet the goals of the No Child Left Behind Act. While the ideas expressed in 
this resource are those of the individual authors and do not represent the official views or positions of the 
U.S. Department of Education, I think you will find the resource to be useful. I offer these white papers 
for your consideration as you work to find better ways to incorporate technology to prepare students for 
success in this 21st century. 

Schools are making tremendous progress under the historic reforms in No Child Left Behind, and student 
achievement is improving. Teachers and students are transforming what can be done in schools by using 
technology to access primary sources, expose students to a variety of perspectives, and enhance the 
learning experience with media, simulations, and interactive software. In many cases, we are 
revolutionizing education and creating learning environments that equip teachers with new tools to 
individualize instruction; engage students in ways never before possible; empower teachers, parents, and 
students with real-time data on student performance; and expand access to resources. 

Too many schools, however, have simply applied technology to existing ways of teaching and learning, 
rather than reinventing themselves around the possibilities technology offers. The results in many cases 
have been marginal. The goal now is to explore and more clearly understand how technology can 
improve instruction and transform what we think of as education. 

I want to thank the thousands of educators and visionaries involved in spearheading the progress in 
education under the No Child Left Behind Act. Your leadership is a beacon toward a new era of 
educational excellence in America that will leave no child behind. 



Sincerely, 



Rod Paige 
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Section I: Empowering Accountability and Assessment Using Technology 



Overview 

States and districts need to manage data resulting from the implementation of No Child Left Behind. 
At the center of accountability systems and empowerment initiatives are a variety of technology 
tools and services. These systems go by a variety of names-business intelligence, data mining, data 
warehousing, decision-support systems, data-based or evidence-based decision-making-but the 
essential idea is the same. They pull together a variety of student demographic and performance 
indicators for detailed analysis. Leaders can explore relationships between indicators and identify 
trends across time or “drill down” into a series of indicators from the district level through the class 
and individual student levels. Essentially, these systems take individual pieces of data and convert 
them into information reports that are useful in guiding decisions and instruction. 

The priority given to student testing in the’Wo Child Left Behind Aet of 2001 requires states to 
rethink their existing strategies for measuring student performance. The costs involved in 
implementing testing, coupled with the changing notion of the utility of the tests themselves, have 
caused a fundamental shift in how states measure student performance. Computer-based, 
technology-based, or online assessments hold the possibility of revolutionizing both how 
assessments are implemented and how student data inform teaching and learning. In addition, these 
systems can be used by teachers to routinely diagnose a student’s mastery of academic standards, 
providing teachers with the information they need to adjust instructional strategies leading up to 
state-administered tests. School, district and state administrators need timely and accurate 
assessment and accountability data. Teachers and students need on-demand access to academic 
diagnostic and performance data. 

This section helps practitioners with: 

• Identifying technology tools and resources that are available to support the accountability, 
student information and data requirements of No Child Left Behind', 

• Demonstrating how to use data to inform decisions; 

• Illustrating how to correlate both data and decision-making to achieve the requirements and 
intent of No Child Left Behind', 

• Promoting state educational agencies’ collaboration on joint development or procurement of 
system components; 

• Promoting the benefits of online assessment for implementing No Child Left Behind 
requirements; 

• Demonstrating how online assessments can improve and strengthen assessments; 

• Providing guidance on how a state can implement an online assessment statewide; 

• Exploring opportunities for multi-state consortia to foster collaboration on the development 
of policies to move toward a system for computer-based assessment; 

• Illustrating how online assessments can inform instruction at the classroom level (diagnostic 
tests, real-time reporting, performance-based, etc.); and 

• Focusing on strategies to make assessment results useful. 

The papers and diagrams in this section are designed to provide school leaders and policymakers 
with information and recommendations with which to help design and implement systemic reforms 
for meeting the requirements of 
No Child Left Behind. 
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A Technology Framework to Support Accountability and Assessment 
How States Can Evaluate Their Status for No Child Left Behind 

Glynn D. Ligon, Ph.D. 

President, ESP Solutions Group 



Overview 

The chief state school officer needs the right assessment and accountability data, right now — and the 
data better be right. Teachers and students need academic diagnostic data — on demand. How does the 
chief know if the state has the information technology in place to accomplish both goals? 

(Information technology is defined as the tools and systems used to share information, e.g., hardware, 
software, networks, and the processes to manage them.) 

Each state’s technology implementation can be unique. Each state can design the education information 
system best suited to its own requirements. Individual schools and districts can make personal choices 
of vendors and software applications. Even with this individuality across schools, districts, and states, 
each one can be aligned to meet their state’s accountability and assessment requirements as well as those 
of No Child Ceft Behind. There is not one technology solution that fits all schools, district, and states. 

States struggle with the ''Education Technology Local Control Conundrum,” which is: How can local 
decision making about technology coexist with the requirements of a standardized state and national 
accountability system? The answer is adopting data and technology standards developed to enable 
interoperability. Interoperability is being able to share data electronically across different software 
applications, different hardware configurations, and different operating systems. 

Each state’s assessment and accountability systems will be judged on whether or not they provide data 
that yield maximum value. Every datum in these systems must be evaluated to ensure that it is worth the 
effort to get it and that it does the job it was collected to do. 

Assessment and accountability systems cannot be successful without extensive technology support. 
Technology to support assessment and accountability requires a comprehensive, standards-based data 
exchange process (interoperability). There must be a smooth, timely movement of data from schools 
to districts to states, and on to the U.S. Department of Education (ED) with appropriate public access 
at each point. In December 2003, ED’s Performance Based Data Management Initiative (PBDMI) 
successfully piloted a process for electronic state-to-federal data exchange. (See 
www.espsolutionsgroup.com/PBDMI.) PBDMI was developed using requirements described by the 
states themselves — requirements that leverage the capacity of each state to report data to the federal 
government or to efforts such as the Broad Foundation’s partnership with USED (School Information 
Partnership, SIP). 

To maximize data driven decision making (D3M), every state’s mandated assessments must be 
administered, scored, reported, and acted upon within a cycle time of weeks contrasted with the 
months states took for less comprehensive assessment programs in the past. Mandated accountability 
reports must be compiled and published in an even shorter time to allow parents to make informed 
school choices, districts to make decisions on school improvements, and states to make school ratings. 
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All this must be accomplished with better data than states had before No Child Ceft Behind. The 
technology for all this is ready to be put into action. 

A state must apply resources to technology-based solutions, because there is no other alternative that 
can deliver assessment and accountability reports on time. This makes technology’s role in assessment 
and accountability that of providing the tools and the infrastructure through which data can flow 
quickly, accurately, and securely. 

Technology and improved information systems will not make all this happen. People will make this 
happen with the intelligent use of technology. Today’s technology tools can help solve a district or 
state’s toughest information challenges. These challenges are described here very simply as getting the 
right data, in the right way, right away, and getting them right in the process. The right data management 
makes this happen. 

States will not be held accountable for the technology they apply to No Child Ceft Behind. States will be 
held accountable for deliverables (e.g., adequate yearly progress determinations, annual report cards, 
diagnostic assessments aligned with academic standards and linked to the state’s assessments, etc.) that 
are dependent upon the efficient use of information technology. 

To assess a state’s status in acquiring and applying technology to the necessary assessment and 
accountability components, the correct questions must be posed, measures that yield valid scores must 
be used, trends across time must be tracked, and benchmarks across states must be established. ED’s 
PBDMI and its data resource, the Education Data Exchange Network (EDEN), will be necessary 
resources. Emergent national standards for education data exchange (e.g.. Schools Interoperability 
Framework (SIF)) will make the flow of data efficient. With these standards, states will be able to 
maintain their individualities and still be able to participate in the nationwide improvement of education 
data for assessment and accountability. 

This paper will not leave states with only this 30,000 foot view of an ideal. Speciflc components for a 
state’s information system, based upon best practices across all states, are described along with 
implementation benchmarks aligned with three familiar sounding ''performance levels” (basic, 
proflcient, and advanced). 

The Vision for Our Data 

Burden, redundancy, expense, lost productivity, lack of comparability, distrust, late reporting and other 
negatives have characterized education data for decades. Today’s goal is to achieve Max Yield Data. 
Max Yield Data simply means data that everyone agrees are worth the effort. Imagine teachers, school 
administrators, program managers, and central office staff all agreeing that a required report yields such 
useful information that all the effort put into the collection and reporting of the data is worthwhile. Max 
Yield Data have been standardized, collected, and presented such that the maximum use can be made of 
them for decision making and reporting mandates. Reaching this goal demands high quality, managed 
accessibility, certiflcation (sign-off that the data are correct and ready to use), interoperability, utiUty, 
affordability, and granularity (a level of detail that allows analysis and interpretation). (Ligon, 2003, Best 
Practice for a Statens Education Information System^ presented to chief state school officers. Lake Tahoe, NV.) 



2 



U.S. Department of Education Secretary’s No Child Left Behind Leadership Summit 
5 



WHITE PAPER 

A Technology Eramework to Support Accountability and Assessment: How States Can 
Evaluate Their Status for No Child Left Behind 




U.S. DEPARTMENT OF EDUCATION 

Secretary’s No Child Left Behind 

LEADERSHIP SUMMITS 

Empowering Accountability and Assessment Using Technology 



The mantra of reformers in the education data world as characterized by the members of the Council of 
Chief State School Officer’s (CCSSO) Education Information Advisory Committee (EIAC) has been 
''collect the data once and use them many times, by many people, for many purposes.” This has been 
the objective of states’ and ED’s efforts to automate data collections and to build data repositories. 

The Steps 

States have followed four steps to successful implementation of significant improvements in their 
technology supporting assessment and accountability information systems. 

1. Evaluate the Current Status: A framework for this evaluation has been defined based upon 
direct involvement with and documentation of major efforts by states. Among the pioneers are 
Florida, Nevada, and Texas. In the latest generation with some new ideas are Iowa, 
Massachusetts, Mississippi, Oklahoma, Nebraska, Virginia, Wisconsin, Wyoming, and others. A 
self-assessment may be a good start, but tapping expertise beyond a single state education agency 
has been the typical approach. The National Center for Education Statistics sponsors a 
personnel exchange that has helped states share their expertise. A common approach has been 
to hire professional consulting firms for formal, independent evaluations. 

2. Identify the Gaps: The difference between the findings of the evaluation and the benchmarks 
established through documentation of best practices across states provides a roadmap for 
improvement. This analysis should include a formal study of the requirements for a state’s 
unique solution. 

3. Develop a Plan: From the requirements study, a formal plan with timeUnes, budgets, and 
implementation benchmarks should be developed. 

4. Implement the Plan: This may require a challenging commitment of resources, continual 
updates, and careful monitoring. 



The Basics 

There are five basic technology-based principles for achieving the Max Yield Data supportive of a state’s 
assessment and accountability requirements. These can be viewed as the technology performance 
standards for supporting successful assessment and accountability systems. 

1 . Get the right data. Validity in an accountability system and specifically in an assessment program 
begins with a precise definition of what is to be measured and what method of measuring it is 
the most appropriate. No Child Geft Behind requires a state’s accountability system to be both 
valid and reliable. In the data world, this means creating common definitions for data elements 
(e.g., a data dictionary) to ensure that all providers of data report comparable data (same 
definitions, codes, and periodicity). Getting the right data begins at the school for most 
education data. Otherwise, nonstandard data (i.e., different definitions, incorrect entry, etc.) can 
be passed faithfully along throughout the information system, perpetuating the problem. 

2. Get the data right. Data quality includes but goes beyond accuracy. As just stated, the data must be 
right from the beginning. All along the way, the data must be correctly exchanged. The proven 
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way to monitor this is with a set of business rules that describe the format, acceptable values, 
missing data options, and logical comparisons to prior reports. Automated processes that verify 
data upon entry contribute significantly to accuracy. (Reducing Cycle Time and Increasing Data Qualily 
for Student A^ssessments^ www.EducationAdvisor.info, Category: Data Driven Decision Making.) 

On the other end, access to data and formal reports must protect the confidentiality of 
individuals and be statistically reliable. (Confidentiality and Reliability Rules for AYP in NCLB , 
www.EducationAdvisor.info, Category: NCLB Requirements.) 

3. Get the data right away. The lag time between testing and availability of the data limits the benefits 
of assessments and is an Achilles heel for assessments and No Child Teft Behind. For any data to 
be useful and used for decision making, they must be current and timely. This is a major new 
accountability requirement for many state assessment programs. Cycle times of months to over a 
year were common prior to No Child Teft Behind. Current assessment programs in which steps are 
linear and sequential (finish testing everyone, clean everyone’s data before proceeding, then 
score all tests at the same time, then analyze results, then report statewide simultaneously, then 
publish all reports together, etc.) may not be the best model for today. 

On-line, web-based testing is an effective best practice. The initiatives in progress in leading 
states should be watched to learn how to take on-line testing to the scale required for widespread 
implementation. 

A major focus is replacing dissemination with access — making results available on demand 
rather than pushing them out to everyone at the same time. (Implications for Collecting, Storing^ 
Retrieving, and Disseminating National Data for Education, Ligon, in U.S. Department of Education, 
National Center for Education Statistics, From Data to Information: New Directions for the National 
Center for Education Statistics, NCES 96-901, 1996.) 

4. Get the data the right way. The right way to get data these days is through an automated process. 
Automated processes can verify data quality and ensure standards are met before data are 
accepted into the state’s information system. States must understand that information exchange 
processes involve complex systems. For example, examining the complete process flow for 
student assessments clarifies that schools, districts, states, vendors, delivery services, printers, 
and web designers ah have crucial roles in the process. Improvements at any single point in the 
flow may not be possible without coordination with other participants. (The Supply Chain of State 
Assessments and Reducing Cycle Time, www.EducationAdvisor.info, Category: Data Driven 
Decision Making.) 

5. Get the right data management. The assessment and accountability systems must be managed well to 
achieve maximum yield from the data. Data management encompasses a broad range of 
administrative activities, infrastructure components, and policy commitments. A long-range plan 
for exchanging data should include policies, funding, human resources, enabling legislation, 
hardware, software, and networking. A poUcy advisory committee, a data provider group (user 
group), and an internal agency coordination group should oversee data management. 



4 U.S. Department of Education Secretary’s No Child Left Behind Leadership Summit 

7 



WHITE PAPER 

A Technology Eramework to Support Accountability and Assessment: How States Can 
Evaluate Their Status for No Child Left Behind 




Empowering Accountability and Assessment Using Technology 



Secretary’s No Child Left Behind 

LEADERSHIP SUMMITS 



U.S. DEPARTMENT OF EDUCATION 




Defining Best Practice 

Many states are managing many processes very well. Only a few are beginning to manage some of the 
more difficult processes well (e.g., reducing cycle time for reporting assessment scores). Even though 
there are 52 very different state-level education agencies mandated to follow No Child Ceft Behind, there is 
an Education Information Technology Framework with 10 components (necessary pieces of the 
infrastructure) that has been identified as representing best practices for a state. In the summer of 2004, 
visits to all 52 state-level education agencies documented their readiness for PBDMI and provided 
insights into each state’s status on key components. (www.espsolutionsgroup.com/PBDMI) 

These components had been identified in 2000 and 2002, when ED’s Office of the Chief Information 
Officer sponsored a series of eight regional and national meetings to define the requirements for 
education information from the school to the district to the state to ED. Exploring how data move 
from the school secretary to the Secretary of Education, these meetings helped build a framework for 
describing a state’s implementation of necessary components for accountability. 
(www.EducationAdvisor.info, Category: Data Quality and Best Practices.) 

These 10 Education Information Technology Framework components can be used as a checklist for a 
state’s self-assessment or as a starting point for a more formal, independent audit process. In the tables 
that follow, each component is defined and illustrated. 

1. Academic and Other Performance Standards: Standards should describe in measurable terms 
the outcomes by which academic performance will be measured. Other areas (e.g., human 
resources, finance, support services, etc.) should also be held accountable using adopted 
standards and aligned measures. (Figure 1) 

2. Data Systems: All required data should be included in the state’s data systems. Statewide 
identifiers for students, employees, courses, facilities, programs, finance categories, etc. should 
be assigned. (Figure 1) 

3. Data Standards: A comprehensive data dictionary should document definitions, codes, and 
formats to be followed statewide. (Figure 2) 

4. Data Quality: Formal processes should verify the quality of data each time they are exchanged. 



6. Automated Data Systems: Data should be collected, stored, and accessed using automated 
systems (e.g., directories, student/school management [student information system, SIS], 
discipline, program management, food services, transportation, library, finance, human 
resources, student performance [assessments], D3M [data driven decision making using a 
decision support system], instructional management). (Figure 3) 

7. Data Consolidation and Access: Timely and easy access to data and reports should replace 
dissemination of reports. (Figure 3) 
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(Figure 2) 



5. Aligned Assessments and Other Measures: Assessments and other measures of outcomes 
should be aligned with the academic and other standards adopted. (Figure 3) 
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8. Electronic Exchange of Records: Records and data should be exchanged electronically among 
automated systems. Electronic systems should be interoperable rather than requiring translations 
at each step. (Figure 4) 

9. Network Connectivity: Schools, districts, intermediate units, and state education agencies 
should be connected for fast and large data exchanges. (Figure 4) 

10. Technology Infrastructure: Assessment and accountability systems should be supported by a 
technology infrastructure built on adequate resources and policy support. (Figure 5) 

NOTE: NCES has published several documents that provide technical assistance to districts and states 
on best practice. Data definitions (Handbooks Online) are available at 
http://nces.ed.gov/programs/handbook/. Others include: 

■ U.S. Department of Education, National Center for Education Statistics. (2001). Technology @ your 
fingertips: A Guide to Implementing Technology Solutions for Education Agencies and Institutions, 
NCES 98-293. Washington, DC: Author. [Available at 
http://www.nces.ed.gov/pubsearch/pubsinfo. asp?pubid=98293] 

■ National Eorum on Education Statistics. (2000). Building an Automated Statewide Student Record 
System, NCES 2000324. Washington, DC: U.S. Government Printing Office. [Available at 
http://www.nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2000324] 

■ U.S. Department of Education, National Center for Education Statistics. (1995). A Pilot Standard 
National Course Classification System for Secondary Education, NCES 95-480. Washington, DC: 
U.S. Government Printing Office. [Available at 
http://www.nces.ed.gov/pubsearch/pubsinfo. asp?pubid=95480] 

■ U.S. Department of Education, National Center for Education Statistics. (1997). Protecting the 
Privacy of Student Records: Guidelines for Education Agencies, NCES 91-521 , by Oona Cheung, 
Barbara Clements, and Ellen Pechman. Washington, DC: U.S. Government Printing Office. 

[Available at http://www.nces.ed.gov/pubs97/p97527/index.html] 

■ U.S. Department of Education, National Center for Education Statistics. (1998). Safeguarding your 
technology: Practical guidelines for electronic education information security. Washington, DC: 
Author. [Available at http://www.nces.ed.gov/pubsearch/pubsinfo. asp?pubid=98297] 

Profiles of Prototypical States 

Figures 1-5 use the accountabihty language of No Child Left Behind to describe implementation 
performance levels for each of the 10 Education Information Technology Framework components. The 
characteristics of various levels of successful implementation are categorized as basic (the legacy of a 
paper-based information world with separate information systems for every purpose; inadequate for a 
state’s current assessment and accountabihty systems), proficient (the state of best practice needed to 
support assessment and accountabihty), and advanced (a higher level supportive of data driven decision 
making and enhanced support for students, teachers, administrators, and pohcy makers). 

These checklists provide a framework for taking Step 1: Evaluate the Current Status of Technology 
Supporting Assessment and Accountabihty Systems. 
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Conclusion 

Accountability is a process. Improvements will come from not only upgrading each of the 10 individual 
components of the Education Information Technology Framework, but also from changing how those 
components work together as an overall system. 

Technology can be used to improve the processes within each of these 10 components. In fact, without 
technology, these processes cannot be fast enough or accurate enough to satisfy the requirements of a 
state’s assessment system, a state’s accountability system, or No Child Left Behind. 

Four steps for achieving the technology support required for assessment and accountability systems 
have been defined. They are: 

Step 1. Evaluate the current status of your state’s information technology for the support of 
assessment and accountability systems. 

Step 2. Identify the gaps between the current status and best practice as defined by the successes 
across all states. 

Step 3. Develop a plan to close the gaps then to reach advanced levels of implementation. 

Step 4. Implement the plan with the best data management practices. 

The framework detailed in the 10 components within this paper provides the starting point for Step 1. 
The benchmarks established through the review of best practices across states provide the starting point 
for Step 2. Peer states can be an excellent resource for Steps 3 and 4. 

Information technology processes and poUcies can be implemented in a unique way within each state. 
Sharing and using the lessons already learned across all states contributes to the best implementation for 
each state. 
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FIGURE 1 Performance Standard 1: Get the right data. 



Component 1. Academic and Other Performance Standards 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



l.a. Performance standards 
have not been developed and 
adopted, or there are 
significant gaps in the 
standards. 

l.b. Other areas do not have 
goals and objectives that are 
monitored and reported. 



Basic Implementation 



l.a. Academic standards describe the skills 
and knowledge, and the performance targets 
for students. Performance standards have been 
developed and adopted for all academic areas. 
Standards are published and readily available. 

l.b. Goals and objectives for all areas 
(students, staff, finance, support services) 
describe the non-academic outcomes targeted 
by the accountability system. Management 
processes may be different across areas and 
programs. 



l.a. An instructional 
management system links 
standards and detailed 
instructional targets with 
instructional resources and 
activities. 

l.b. Management plans link 
resources, target dates, and 
dependencies. Plans are 
monitored and formal reports are 
published. 



Component 2. Data Systems 

Proficient Implementation Advanced Implementation 



2. Data systems are in 
separate “stovepipes” using 
different file standards without 
the capacity to share data 
across areas, offices, and 
programs. Decisions about 
which data to collect are made 
independently by districts, 
programs, departments, etc. 



2. A comprehensive process has been 
implemented to ensure that all required data 
are collected. Automated systems are 
implemented and aligned to collect and 
manage the data. Data systems are available 
for the full range of content areas (e.g., course 
data, program participation data, enrollment 
data, graduate follow-up data, assessments 
(statewide, diagnostic, college entrance, etc.), 
background and demographic data, staff data, 
financial data, etc.) Statewide identifiers for 
students, employees, courses, facilities, 
programs, finance categories, etc. should be 
assigned. 



2. A data driven decision making 
(D3M) system ensures that the 
data required are collected, 
stored, and accessible when 
needed for all approved purposes. 
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Figure 2 Performance Standard 2: Get the data right. 


Component 3. Data Standards 


Basic Implementation 


Proficient Implementation 


Advanced Implementation 


3. Individual programs and 
offices determine the data 
element definitions and code 
sets they use. 


3. Data standards are adopted to establish a 
common definition for all data collected and 
reported. 

3. a. A data dictionary is published to inform 
everyone of the up-to-date standards. All 
programs and offices comply or can 
crosswalk to it. 

3.b. Alignment with national standards and 
federal requirements ensures that when the 
data are reported to other entities that they 
are comparable and usable. 


3. a. The state’s data standards are 
available on the web in an 
electronic format that can be 
downloaded and imported into 
databases and applications. 

3.b. Every database application has 
the capacity to meet data exchange 
standards (e.g., SIF) 


Component 4. Data Quality 


Basic Implementation 


Proficient Implementation 


Advanced Implementation 


4. Data quality is not 
examined formally. Schools, 
districts, programs, and the 
state accept the data which 
are available as the best that 
can be provided with minimal 
feedback on the quality. 


4. Data quality is clearly defined, monitored, 
and required. Data quality is the 
responsibility of everyone at all levels of the 
education enterprise. Edit checks are 
performed each time data are exchanged. 


4. Data driven decision making is 
practiced with confidence in the 
data and reliance upon the data. 
Providers of the data rely upon 
them and ensure their quality. 
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Figure 3 Performance Standard 3: Get the data the right way. 



Component 5. Aligned Assessments and Other Measures 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



7. a. Student performance 
measures predate adoption of 
standards or are not 
customized to the state’s 
standards. 

7.b. Other areas do not have 
formal performance measures. 

7.C. Reporting requirements 
for grants, federal funding, 
etc. are met at a minimal 
level, possibly with whatever 
data are available already. 



7. Assessments and other measures are 
aligned with the accountability measures 
implemented. 

7. a. Items and objectives in the state 
assessments are mapped to the state 
academic standards. 

7.b. Performance measures for other areas 
(e.g., staff, finance, support services) are 
aligned with the goals and objectives 
targeted by the accountability system. Other 
areas have formal performance measures. 

7.C. Performance measures for grants, 
federal funding, etc. are met using data 
directly from statewide performance 
measures or from measures aligned directly 
with the requirements. 



7. a. State assessments measure the 
instructional targets within the 
academic standards with 
established validly and reliably. 

7.b. Performance targets for other 
areas are sufficient to evaluate 
success and to support program 
improvement. 

7.C. Requirements for grants, 
federal funding, etc. are fully met 
with data sufficient to evaluate 
success and to support program 
improvement. 



Component 6. Automated Data Systems 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



8. a. Paper forms are used to 
collect aggregate statistics. 
Forms converted to the web 
are not redesigned for 
efficiency. Validation of 
entries is minimal. 

8.b. “Stovepipe” data files are 
used. 



8. Automated data systems collect and share 
the data efficiently. 

8. a. Collection systems are electronic, 
typically networked (on-line). 

i. The periodicity (as-of dates and time 
periods represented) of the data are clear. 

ii. Longitudinal data points are available for 
describing trends. 



iii. Entries are verified and error messages 
provided. 



8. a. Individual student and staff 
records are exchanged with the 
state where statistics are 
calculated. Web-based reports 
provide reports to districts and 
schools. 

8.b. Programs and offices at the 
state level access the data they 
need and are authorized to use. 
Automated updates of their files 
occurs as data are verified from 
schools and districts. 



8.b. The systems and their data are 
interoperable (i.e., capable of moving from 
one system to another without translation). 



8.C. Permanent, unique identifiers are 
assigned to students and staff to ensure 
matching of records. 
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Component 7. Data Consolidation and Access 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



9. a. “Stovepipe” data files 
exist. 

9.b. Aggregate statistics are 
compiled by schools and 
districts and reported. 

9.C. Individual programs and 
offices manage their own data. 
A comprehensive data access 
and use policy is not adopted. 



9. Data consolidation and access are efficient. 

9. a. A data repository, warehouse, etc. 
consolidates the data in a format that is well- 
documented. 

9.b. Linkable individual/unit records (e.g., 
students, staff, finance, programs) with 
unique, permanent identifiers allow separate 
pieces of data for the same individual to be 
linked and for related individuals’ data to be 
correlated. 



9. National standards and best 
practice across the states have 
been incorporated into the state’s 
information system. Longitudinal 
analyses are possible using 
individual IDs, common course 
numbers, and standard directory 
data elements across years and 
files. 



9.C. Access to the data is managed carefully. 

i. Authority to access data and reports is 
defined for individual users related to specific 
data. 



ii. Reports meet the varied needs of the users 
(e.g., actionable accountability reports, 
diagnostic reports, ad hoc queries) 
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Figure 4 Performance Standard 4: Get the data right away. 



Component 8. Electronic Exchange of Records 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



5. a. Schools and districts 
assign local student IDs if they 
choose. 

5.b. Schools contact prior 
schools by mail, phone, or e- 
mail to request transcripts, 
which are faxed or mailed. 
Mobile students are retested 
or services delayed until 
records arrive. 

5.C. Data are entered 
separately into each software 
application (or paper records). 
Changes are made multiple 
times to each application. 



5. Electronic exchange of records avoids 
printing and/or re-entry of data across systems. 

5. a. A student locator function allows schools 
to look up records for new students to find 
prior education records and student IDs. A 
unique, permanent student ID is assigned to all 
students and used in all data exchanges. 

5.b. Student records (transcripts) move 
electronically between schools to speed 
placement and avoid re-assessment of mobile 
students. 

5.C. The interoperability of systems allows for 
the immediate electronic exchange of data in 
all systems whenever updates are entered into 
one system. 



5. a. A web-based look-up 
application allows new students 
to be assigned IDs and mobile 
students’ IDs to be verified. 

5.b. A system is in place to move 
student records upon request 
from one school’s database to 
another’s. A statewide course 
numbering system is 
implemented. 

5.C. Software applications are 
interoperable (by SIF or a custom 
exchange system) so each entry is 
shared across all systems. 



Component 9. Network Connectivity 

Basic Implementation 

6. Schools and districts have 
disparate wide area network 
capability, some with dial-up 
or under-sized capacity. 

better speed adequate for their 
peak work loads. 



Proficient Implementation 



Advanced Implementation 



6. All schools and districts have network 
connectivity to each other and the state at 
speeds and capacity adequate for their normal 
work load. 



6. Schools, districts, 
intermediate units, and the state 
are all connected to the Internet 
(or private network) with T1 or 
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Figure 5 Performance Standard 5: Get the right data management. 



Component 10. Technology Infrastructure 



Basic Implementation 



Proficient Implementation 



Advanced Implementation 



10. The technology 
infrastructure is aging. 
Plans for required upgrades 
either have not been 
adopted or have not been 
implemented. 



10. The technology infrastructure (the 
architecture and management of hardware, 
software, network, and data) is adequate. 

10. a. The technology infrastructure has adequate 
capacity for storage, compilation, and transfer of 
data. 

10. b. Confidentiality and security are ensured 
through both physical and process controls. 

10. c. The technology infrastructure is supported 
by the necessary policy, funding, human 
resources, and security. 



10. The state’s education agency 
leadership and staff have 
developed a long-range plan for 
information technology and 
architecture. This plan ensures 
that changes in requirements and 
technology are addressed on an 
on-going basis. The plan is fully 
implemented. Policy and user 
advisory groups actively monitor 
and support the data 
management processes. 



10. d. The state’s legislature, education board, 
and education agency leadership have adopted 
the goals, enabling legislation, and funding. 
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How States Can Use Information Technology 
to Support School Improvement Under NCLB 

Chrys Dougherty, Ph.D. 

Director of Research, National Center for Educational Accountability 

The No Child Left Behind Neds assessment and reporting provisions cast a spotlight on the value of 
information for school improvement. At the same time, the law’s accountability provisions push 
schools and districts to accelerate the pace of improvement. This calls for policy leaders to shift the 
state education agency’s primary mission from compliance monitoring to that of a State Education 
Information and Improvement Agency. To make this happen, the state’s political leadership must be 
willing to change budgetary and staffing priorities to give the agency the capacity to support its 
expanded data collection, analysis, and reporting responsibilities. 

The discussion of information technology should begin with the products to be created. By 
collecting the right data, state agencies can promote the creation of three basic information 
products: 

School performance reports that identify effective schools and highlight the improvement 
opportunities that these schools demonstrate. Educators serving the disadvantaged often believe 
that they are producing the best results reasonably possible with those students. Highlighting 
schools that are doing better with similar or more disadvantaged students can change those 
perceptions. Those reports can also point parents, educators, and community leaders to the schools 
whose practices they should learn more about.^ 

Best practices reports that examine how the practices of consistently high-performing schools 
differ from those of average- and low-performing schools. These reports use surveys, site visits, and 
collection of documentary evidence and artifacts to examine what is going on in schools. They 
answer the question, ''What are the high-performing schools doing to get their results? How is that 
different from what is happening in average- and low-performing schools?” 

Diagnostic reports for educators on each student’s mastery of specific skills and on which teachers 
have been most successful at getting their students to master those skills. 

While the desirability of these three kinds of reports may seem obvious, what is less obvious is the 
critical role of state data collection in making these reports possible. Consider that: 



^ For example, the Just for the Kids Opportunity Gap Charts show the difference between a school’s performance in 
each grade and subject and that of the highest-performing schools serving equally or more disadvantaged student 
populations. These charts are available on the U.S. Department of Education-sponsored Web site www.schoolresults.org 
and on the Just for the Kids Web site www.just4kids.org . The Just for the Kids site also contains Best Practices 
information and reports showing the accomplishments of schools that have been consistently high-performing in 
multiple grades, subjects, and years. The Education Trust’s Dispelling the Myth charts on www.edtrust.org are another 
excellent example of this type of report. 
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1) Good best practice investigations depend on accurate identification of consistently high- 
performing schools. 

2) Accurate identification of high-performing schools requires a longitudinal student 
information system that can follow the academic progress of students over time and identify 
how long each student has been enrolled in the school.^ 

3) When students change schools and districts, diagnostic information should follow the 
student. 

In other words, the foundation of complete school reports, best practices investigations and 

diagnostic information for mobile students is a statewide longitudinal student information system.^ 

To assist in the creation of the three kinds of reports, a State Education Information and Improvement 

A.gencj can take the following three actions: 

1 . Create a statewide longitudinal student information system; 

2. Convene a task force on investigation and dissemination of best practices; and 

3. Make diagnostic information available to educators. 

1 . Create a Statewide Longitudinal Student Information System 

A complete longitudinal student achievement information system should have at least nine components 

■ A statewide student identifier that makes it possible to match individual student records 
across databases and years, converting '‘snapshot” to longitudinal information. 

■ Student-level enrollment, demographic, and program participation information 

collected at a different time from the state test so the focus at the time of collection is on 
getting the student information right, not on test administration and security. 

■ Student-level test data that is stored permanently by the state so that it can be matched 
with later test results of the same students. 



2 Without information on the same students’ prior test scores, the “value-added” components of a school or program 
cannot be assessed. When comparing schools’ achievement levels, it is also important to know how long students have 
been enrolled in the same school. Otherwise the analysis may be based on students whose achievement is mostly the 
product of other schools. See Robert Meyer, Value-Added Indicators of School Performance: A Primer. Economics of 
Education Keview, 16(3), 1997. 

^ Section 1111(b) of the No Child Ceft Behind Act that states “may incorporate the data from the assessments under 

this paragraph into a State-developed longitudinal data system that links student test scores, length of enrollment, and 
graduation records over time.” 

^ Chrys Dougherty, “Nine Essential Elements of Statewide Data-CoUection Systems,” Education Commission of the 
States, 2003, available for download on www.nc4ea. org . 
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■ Information on untested students — a No Child Ceft Behind requirement that requires 
accurate accounting for all students enrolled in tested grades at the time of the test. Absent 
and exempt students should be accounted for along with the reason for each student’s 
exemption so that trends in test participation of different student populations can be tracked 
over time. 

■ Student-level course completion information to see how many students are taking 
challenging academic courses in middle and high school. 

■ Student-level SAT, ACT, and AP exam results to show how many students from 
different backgrounds are participating in these exams and demonstrating readiness for 
college. 

■ Student-level dropout and graduation information that flags students who leave the 
state’s public education system and accounts, as well as possible, for where these students 
went. 

■ Ability to connect K-12 and college records to see how the state’s high school graduates 
fare in college. 

■ A state data audit system that uses both statistical checks and occasional site visits to 
review school district records. 

School districts can develop longitudinal student information systems with all of these elements, but 
without a similar system at the state level they lose the advantage of statewide comparisons. Who 
wants to be limited to a single school district in flnding the most successful schools?^ 

Given the cost of developing such a system, state policy leaders must be prepared to explain the 
benefits. Those benefits include: 

■ Parents can have better information on students’ academic progress and can distinguish high 
'Value added” schools from those that coast on the success of entering students. Parents of a new 
sixth-grade student, for example, can have access to better reports on the success of the school’s 
sixth grade. 

■ Educators can learn from the practices of the most successful schools in the state. Educators in 
schools with highly disadvantaged or mobile student populations can be shown where others 
have succeeded with similar students. Reports on the academic achievement of continuously 
enrolled students can erase the perception that low test scores are explained by the performance 
of students who just showed up. Middle and high school educators can assess how their schools 
perform with students who enter at different levels of academic preparation. For example, how 
does our high school do with students who were proficient on the state’s eighth grade exam? 
How successful are we compared to other high schools when working with students who had 
failed the eighth grade exam? 



^ Direct nationwide comparisons are not possible because states give different tests. However, a multi-state directory of 
high-performing schools can be created once these schools are identified in each state. 
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■ School district leaders can analyze the success of their schools based on the academic 
preparation of entering students. 

■ Researchers can evaluate schools and programs based on student academic growth and the 
length of student enrollment. They can assess strategies that work well with mobile student 
populations. They can follow students from one level of education to the next to evaluate 
the schools’ long-term success. 

■ State policy leaders can have better information on which policies are working. 

A longitudinal student information system relieves school districts of having to reconstruct, every 
time students are tested, all of the background information on every student: each student’s 
ethnicity, economic disadvantaged status, English Language Learner status, special education 
participation, migrant status, and gifted and talented enrollment. For previously enrolled students 
that information needs only to be updated. The inaccuracies resulting from having to rebuild all of 
the information from scratch are well known.^ Inaccurate information on student membership in 
disaggregated groups can lead to faulty identification of which schools made Adequate Yearly 
Progress (AYP). 

2. Convene a Task Force on Investigation and Dissemination of Best Practices 

The State Education Information and Improvement A.gen(^ should work with other entities in the state — 
nonprofit research organizations, universities, and business-education alUances — to organize the 
process by which effective practices are researched and disseminated. 

'‘Best practices” has sometimes been used by advocates as a label to market their favorite 
educational philosophies. What distinguishes the practices discussed here is that they are validated by 
empirical research or data in one of two ways: either they are found by experimental research to be 
effective, or they are found to be present in high-performing schools at much greater frequency than 
in average- or low-performing schools.^ 

The new generation of best practices reports are distinguished by four additional features. First, in 
states with longitudinal student information systems, the reports are grounded in better data on 
high-performing schools. Second, the reports are based on a conceptual framework that provides 
the following information on each practice: 

■ Nature of the practice. 

■ Level of the practice: district, school, or classroom. (For example, each level may have 
different responsibilities in developing and implementing the practice.) 



^ These inaccuracies are discussed in Chrys Dougherty, “States Must Improve Data for No Child Left Behind,’’ Education 
Assessment Advisor^ August 2002, available under “Data Collection” on the Web site www.nc4ea.org . 

^ The latter approach to best practice research is more common since true experimental research is rare in education. 
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■ Evidence that the practice is more commonly found in high- than in average- or low- 
performing schools, districts, or classrooms. 

■ Supporting conditions that make it more likely for the practice to be implemented and 
succeed. For example, a relatively clear set of state academic standards can facilitate the 
development of a clear and specific curriculum at the district level. 

Third, the reports are designed to be easily usable by practitioners, providing examples of what these 
practices look like in specific high-performing schools.^ Finally, the reports are accompanied by self- 
audit tools that enable educators or involved laypersons to assess the extent to which each practice is 
present in their own classroom, school, or district.^ 

3. Make Diagnostic Information Available to Educators 

The increase in state testing is providing information on the specific academic skills mastered by 
each student. This information should be supplemented by local assessments that cover more skills 
than a state test of reasonable length can address. These local assessments can be given during the 
course of the school year so that teachers pinpoint their students’ academic strengths and 
weaknesses in a timely manner and can intervene quickly. This rich set of diagnostic information on 
the achievement of each student can be organized into a database that provides teachers and school 
administrators detailed information on which students need additional assistance.^^ 

The student diagnostic information can be organized by classroom in two ways: 

1. By the student’s classroom in the tested year, so that teachers and school administrators can 
monitor student progress and see which teachers are doing the best job of teaching which 
academic skills. For example, if Mrs. Jones does a better job teaching fractions than Mrs. Smith, 
then perhaps Mrs. Smith can be paired up with Mrs. Jones to learn what she is doing. Mrs. Smith 
can reciprocate if there is an area of teaching in which she is stronger than Mrs. Jones. 

2. By the student’s classroom at the beginning of the folloiving school year, so that teachers get a detailed 
picture of the academic skills of their incoming students and can plan instmction accordingly. 

The state agency can assist in the design of a common architecture for these databases so that 
software written for one district can serve the needs of another. In addition, the state can write 
software to put its own test results into that database and supply the information to the districts. 



^ For example, if “High-Performing Schools do Frequent Diagnostic Assessments,” the reports provide examples of 
what these assessments look like in specific districts and schools and a detailed discussion of how the assessments are 
used by principals and teachers in collaborative efforts to assist students who are having difficulty. 

^ The Web site www.just4kids.org contains a set of best practices reports that have all of the features described here. 

The database can also show which students have mastered the curriculum early and are ready to move on so that 
advanced students can receive appropriate instruction at their level. The Northwest Evaluation Association 
( www.nwea. org) has specialized in developing computerized diagnostic testing systems that assess the skills and 
academic growth of students, including those who are performing well above or below grade level. They have a large 
item bank from which assessments aligned with a specific state’s standards can be built. 
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Finally, the state may wish to have a common set of data analysis software written for all of the 
districts so that everyone is able to cut their data in the same ways, and a teacher or administrator 
moving between districts does not have to learn an entirely new data system. 

Questions for State Education Policy Leaders 

1. How do the state education agency’s mission statement, staffing patterns, and budgetary 
priorities reflect its new mission as a state education information agency? 

2. How do the agency’s staffing patterns and budgetary priorities enable the agency to carry out the 
three specific activities — collect the right information, promote best practices, and make 
diagnostic data available to educators — that have a significant potential to help schools meet the 
ambitious improvement goals established under the No Child Ceft Behind Act} 

3. When is the Web-based state school report card available to the public for the previous spring’s 
testing? What constraints are delaying the availability of this information to parents, educators, 
and the public? 

4. How are the policy leaders in the state supporting the state agency’s capacity to collect the right 
information, make timely diagnostic information available to educators, and make timely school 
reports available to the public? 

5. What process exists in your state to evaluate the usefulness of the state’s Web-based school 
reports to educators, parents, and other members of the public? 

6. What infrastructure exists in your state for investigating and disseminating best practices and 
assisting low performing schools in implementing these practices? 

7. What infrastructure exists in your state to make diagnostic information available to educators? 
Who is working on a solution to this problem? 

Web sites 

State Data Collection 

www.nc4ea.org 

http:/ / evalsoft07.evalsoft.com/pbdmi 

School Reports 

www.schoolresults.org 

www.just4kids.org 

www.edtrust.org 
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Best Practices Reports 

www.)ust4kids.org 

Diagnostic Testing Systems 

www.nwea.org 
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Improving achievement through Student Data Management 



XPLANATjONS® by xplane® 



On average, there is little aggregation of student data in today’s school systems. Information is siloed, redundant and difficult to 
share. The technologies used — if any — are aging and frequently incompatible. An ideal state has complete aggregation and 
alignment. It is easier to ensure that students meet challenging standards, teachers target instruction, parents know teachers are 
helping their children, school districts know how to allocate resources effectively and the government knows how schools are doing. 



1. The average state: Isolated silos 
of information prevent everyone 
from seeing the ‘Big Picture.’ 



2. The ideal state: A Total Information Management Tooi 
(Data Warehousing) wiii aggregate previousiy siloed data 
and create a variety of reports for any audience. 



3. The result: These reports inform 
instruction, resuiting in continuous 
student improvement. 




(Student Information System^/ 



Federal 



Student 



The tool creates reports 
that everyone can access 



Transportation 



Special Education 



Transportation 



Finance 



Food Services 



(Student Information Systems) 



These reports help inform instruction 
and reallocate resources effectively 



Food Service 



Assessment 



An interoperable system securely 
connects all information and technologies 



Finance 



Federal 



Parent&’ 



Special Education a 



Studenlfr 



Distri ct Te achers 



Other 
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continuous collection of information for future improvements 

©2004 XPLANE.com® 




Steps for Ensuring Data Quality 



Data quality is more than accuracy and reliability. 
High levels of data quality are achieved when 
information is valid for the use to which it is 
applied and when decisionmakers have 
confidence in and rely upon the data. 

Implement these steps organization-wide 
to increase and maintain data quality. 



0 Are data appropriately analyzed and reported? 



Disclose all conditions 
affecting interpretation 
of the data. 

Present conclusions 
fairly within a context 
for interpretation. 

Review data with those who 
have a stake in the results. 



Ensure analysis techniques meet 
the requirements for proper use. 



0 Are data verified 
^ and compared? 



Q Run audit reports for review 
by experts with knowledge 
of reasonableness. 

Q Compare data to past runs, 
standards, or similar groups. 



Protect FERPA 
confidentiality rights. 

Publish technical reports 
or make available files 
with detailed data for 
verification. 



Q Check data exchanges, 
crosswalks, and 
translations for integrity. 

g Verify all calculations 
and conditional rules. 



rte^ 

O 

o 



This student 
group needs... 




Is process well documented 
and communicated? 












y Use checklists and 
signoffs for key steps. 

^ Run sample data and verify. 



^ Conduct on-site reviews 
during the process. 



4 ^ uona 
durin 



^ Ensure all 
requirements 
are available 
(e.g., computer 
hardware, 
software, 
network, etc.). 



^ Review design by peers, 
agencies, and staff. 

y Preprint all available 
data. Limit times data 
are entered. 

yf Ensure target dates are 
reasonable and clear. 



y/ Provide documentation 
for data providers and 
data processors. 

yf Provide a data dictionary 
and format specifications. 



yf Use most automated/validated 
level of data entry possible 
(e.g., codes in an automated 
application vs. paper forms). 



yf Ensure problems 
are reported, 
documented, 
corrected, and 
communicated back 
to the source of the 
problem or report. 



yf Provide training and 
certification for data 
providers and all 
new staff. 

y Provide immediate 
help for data 
providers. 



yf Use random checks 
during production. 



appiicaiion vs. paper Tormsj. ^ Pup maintenance 

before all production. 

y/ Automate entry verification at the Verify off-hour 
earliest levels (e.g., upon key stroke maintenance and 
vs. from printed audit report). staff availability. 




Data-driven decisions 
made with confidence 

Comparable data; 
interpretable 
beyond local 
context 

5 Quaiity 

Valid data 
consistent with 
construct being 
described 

Reliable data 
independent 
of collector 

Accurate data 
consistent with 
definitions 

4 Vaiid 

Accuracy 
achieved for 
decision making 

Designation of 
official data for 
decision making 

Periodicity 
established for 
collection and 
reporting 

3 Official 

Data combined, 
aggregated, 
analyzed, 
summarized 

Data collected 
by some at 
some times 



2 Available 

Inconsistent 
forms of 
measurement 



m Are requirements 
known? § 




y/ Compare policy, 
regulation, and 
procedures with 
instructions given to 
data providers, 
collection forms, and 
code in computer 
programs. 



^ Ensure all personnel are 
knowledgeable, certified, and 
trained for their assigned tasks. 

y/ Include data providers 
and data processors in 
decisions to establish 
what is feasible. 




ESP 

solutions 

group 



Want to learn how to improve your data quality? 

ESP Solutions Group conducts comprehensive data quality audits for local, 
state, and federal education agencies. We can assess your data processes, 
identify your strengths and weaknesses, and make practical recommendations 
on how to improve the data quality across your entire organization. 

For more information on ESP’s data quality services, visit 
www.espsolutionsgroup.com/dataquality. 



y/ Follow an established 
change-management 
process. 

y/ Comply with 

professional standards 
for data collection, 
analysis, and reporting 



The Hierarchy 
of Data Quality 



Data 

unavailable 



L 1 Data Defined 

Bad data 

-1 Invalid 



As stages are completed, data quality and usability 
increase. Flow do your data measure up? 
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A Technology Framework for No Child Left Behind Success 



Each state can create a powerful technology infrastructure through the successful 
implementation of these ten components. You need the right data supporting the right goals 
captured the right way. How does your framework currently rank? Check the status of your 
state’s implementation within each component. The path is clear— more advanced technology 
will ensure better success with No Child Left Behind. 



Data Systems 



Your State 
Department of 
Education 








Disparate data 
systems, file 
standards, and 
data-collection 



Data systems 
implemented and 
aligned around 
required content areas 



Data Standards 

Each office with 
unique definitions 
and codes 

Standards and 
definitions published 
in system-wide data 
dictionary 

Data standards 
electronically 
exchanged and 
adopted by all 
applications 



Academic / 
Other 

Performance 

Standards 





Data driven decision 
making (D3M) system 
implemented 



Management plans linked 
to standards, instruction, 
and resources 



If... 

Then... 



Standards not 
developed or adopted 



Standards and management 
processes developed, 
adopted, and published 




Long-range plans, long-range ^ 
vision by education agency 
leaders, and involvement of 
policy and user groups 



Aging infrastructure: 
no plans to upgrade 




Adequate infrastructure: storage 
capacity, security controls, and 
support (policy, funding, 
and human resources) 



Technology 
Infrastructure 

(architecture and management of 
hardware, software, network, and data) 



Electronic records 
follow students, SIF 
standards, application 
interoperability 

Electronic exchange 
of records; system 
interoperability 



Data keyed from 
paper records 



Electronic Exchange of Records 
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Basic Implementation 



Proficient Implementation 



Advanced Implementation 



Data Quality 

Data quality 
not monitored 



Data quality is clearly 
defined, monitored, 
and required 



D3M practiced 
confidently; data 
providers monitor and 
rely on quality data 




ESP 

solutions 

group 



Aligned 
Assessments / 
Measures 




Performance 
measures predate 
standards 



Assessments 
aligned with up-to- 
date standards 



Assessments 
measure targets 
reliably 








Automated 

Data 

Systems 



Individual-level statistics 
calculated and shared via 
web-based reports 



Automated systems collect 
and share data for clearly 
defined timeperiods 



Collection via paper 
forms generates 
“isolated” data files 



National standards incorporated into 
repository; longitudinal analysis 
enabled by standards 



24/7, fast 
connections 



Constant, adequate 
speed connections 



Inconsistent access; dial- 
up or under-sized capacity 

Network Connectivity 



Data repository consolidates data; 
security is managed at user 
and specific data level 




Aggregate statistics compiled by 
schools and districts; no data 
usage/access policy 



Data Consolidation, 
Security, and Access 



Want to learn how your SEA can move from one level to the next? ESP Solutions Group can provide your SEA with extraordinary insight into K-1 2 education data systems and 
psychometrics. Our team is comprised of industry experts who helped pioneer the concept of “data driven decision making.” ESP personnel have advised all 52 state education 
agencies as well as the U.S. Department of Education on the practice of K-1 2 school data management. We are regarded as leading experts in understanding the data and 
technology implications of the No Child Left Behind Act (HCIB), Performance Based Data Management Initiative (PBDMI), and the Schools Interoperability Framework (SIF). 

Get your free copy of the accompanying whitepaper at www.espsolutionsgroup.com/sea 














Paper-based testing: Slow, expensive, not useful for instruction 
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One of the major requirements for NCLB is annual assessment of students in core subjects beginning with reading and math. However, the 
traditional paper-based approach has several downsides — including untimely feedback which takes 4-6 months to generate results, high costs 
associated with administrative overhead and use of multiple resources to duplicate, administer, collect, collate, code, score and analyze the data. 




1 . Teacher hands out tests and 
packages them for scoring 
after completion. 



2. A couple months later 
the students go on 
summer break. 



3. The test results come 
back during the summer 
- and only the teacher and 
I the principal see them. 



Parents receive their 
children’s test results 
from last school year. 



4b. Teacher files away 
test results from last 
year’s students. 



Total time 
to test results: 



Informing 

instruction: 



Transport, 
security, 
paper costs: 



5. The new teacher has no 
information about the 
students, causing the 
teacher to start fresh. 



Expensive 



Bottom Line: Costly 
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Online testing: Fast, inexpensive, useful for instruction 
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By taking a web-based approach, states and schools use tests to focus on improved instruction through rapid assessment 
and feedback — not only to meet annual accountability requirements but also to improve daily instruction. Everyone benefits. 
Teachers tailor lessons around child needs. Parents know their child’s strengths and the areas needing improvement. 



starting the transition 
from traditional to online 




1 . Technology is analyzed to check 
for sufficient hardware, level of 
technology and bandwidth 




2. Any necessary upgrades are made 




3. Internal training begins. Teachers and 
administration are shown how applications 
work and how results can be utilized 




4. Students instructed on how to use 
test tools and begin testing online 





Bottom Line: Priceless 



Transport, 
security, 
paper costs: 



Inexpensive 



Informing 

instruction: 



100% 



3. The teacher is able to 
tailor individual instruction 
according to needs. 



Total time 
to test results: 



24 Hours 



2. Results are returned in less than 
24 hours so the teacher can 
review results with students. 



1. Class takes test online 
through a secure server. 
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The benefits of online Individualized Education Programs (lEP) 
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Traditional lEP 

lEPs are designed to create the 
best educational environment for 
special needs children. And yet, 
the traditional lEP is wrought 
with excessive paperwork. Here 
is a look at where some of the 
problems occur: 

Special educators spend at least 10% 
of their time on paperwork, away from 
individualized instruction. Multiple 
copies must be filled out and mailed to 
the lEP team. Innaccuracy is common. 




Forms are filled out 
and delivered to the lEP team 



Online lEP 

By moving to a web-based lEP, 
paperwork decreases and focus 
on the child increases. Special 
educators stay in the field instead 
of being lost to other less paper- 
intensive positions. 

Online lEP requires 50% of the time 
previously spent on paperwork, 
allowing special educators to focus on 
a child’s Instructional needs. 

Accuracy is also greatly improved 
through online applications. 



Relevant Information Is updated 
and distributed electronically 



Many of these forms take two days to 
complete, so they are not updated 
frequently. 

The lEP Is reviewed annually by the 
lEP team to test the child’s progress. 
Boxes of forms must be reviewed. 
Once again, innacuracy is inevitable. 



Student Information is tracked in a 
shared database: Notices are 
prepared and sent online, multiples 
are eliminated, reports are simple to 
create and students are easily 
transferred to different schools when 
necessary. 

Accurate information Is quickly, easily 
accessed during lEP reviews. 





Case worker 



Parents 



District 






m 
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Traditional lEP is inefficient 



Online lEP is efficient 













Section II: Increasing Options Through e-Learning 



Overview 

E-learning is a broad term reflecting a wide range of educational courses and supplementary digital 
materials delivered anytime and anyplace using technologies. The major difference between a 
traditional course and an online course is the physical and temporal separation between the student and 
the teacher, and often among members of the class. 

E-learning enables educational and professional development opportunities to occur any-time, anyplace, 
and can supplement and enhance a school’s ability to deliver quality educational experiences for all 
students across a broad spectrum of needs. 

Because e-learning leverages the power and flexibility of digital tools, and the timeliness and immersive 
and adaptive capacity of digital materials, it can be used to create personalized learning opportunities all 
along the instructional continuum. Erom enhancing face-to-face classrooms to providing complete 
virtual degree programs, e-learning can allow teachers to reach students regardless of distance, 
disability, medical conditions or time constraints. E-learning opportunities are increasing in elementary, 
middle and high schools, as well as in higher education, as there is growing consensus that quality e- 
leaming opportunities should be made available to all learners. 

This section helps practitioners with: 

• Providing states with virtual school policy frameworks and models; 

• Providing assistance on how e-learning supports provisions in 

No Child Left Behind on choice, supplemental services, professional development and training; 

• Promoting virtual schools as a viable opportunity for students; and 

• Demonstrating examples and best practices for schools in transforming education using e- 
learning. 
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e-Learning Frameworks for NCLB 

Susan R. Collins, KCH Strategies 



Educators must embrace e-leaming solutions if they 
want to ensure that every student has a quality educa- 
tional experience. But before e-leaming can achieve 
widespread acceptance in public schools, educators 
and policy makers must expand their notion of educa- 
tion to include online courses and digital materials 
used to enhance classroom instmction. 

The No Child Left Behind (NCLB) Act requires a rig- 
orous academic curriculum and highly qualified teach- 
ers — and it holds public schools and teachers 
accountable for student success. E-learning solutions 
can supplement and enhance a school’s ability to de- 
liver a quality educational experience for all students. 

This paper will describe e-learning solutions to com- 
mon problems facing educators, outline current barri- 
ers to the widespread use of e-learning, and describe 
characteristics of an “ideal” state environment for 
e-learning. 

PART 1 : Understanding the power of e- 
learning 

The following scenarios describe real problems from 
the education trenches and the e-learning solutions that 
resolved them. 

Rural Pennsylvania. The German language teacher re- 
signed in June. In mid-August, despite the principal’s 
best efforts, there’s still no teacher for German II. How 
can the class’s 1 5 seniors meet their foreign language 
graduation requirement? 



An online German II class solved the problem. The stu- 
dents ^dttended class^^ in the computer lab^ submitted 
their homework and took exams online^ and participated 
in online discussions. With a stajf member acting as a 
mentor, the students kept up with their studies and con- 
tinued learning. At the end of the year, stajf and students 
judged the replacement class a success — thanks to the stu- 
dents^ efforts, the quality of their online course, the skill 
of their online instructor, and the support of their in- 
school mentor. 

Small-town Texas. Each year there are a few math stu- 
dents who really should be taking Advanced Placement 
Calculus — but never enough to warrant offering the 
class. How can the math department continue to chal- 
lenge its best students? 

An online Advanced Placement class solved the problem. 
The students who needed the challenge of an academically 
rigorous mathematics curriculum took the AP course 
online, completed it successfully, and went on to pass the 
AP exam. By arranging for the online course, the school 
expanded the students^ learning opportunities and also 
improved their chances of meeting college entrance re- 
quirements. 

All over the country. The seventh-grade social studies 
curriculum is ambitious and the teacher is highly capa- 
ble. As in most classes, however, the students represent 
a wide range of academic preparation and educational 
ability. How can the teacher help every student succeed? 

A variety of digital materials helped the teacher provide 
additional opportunities for individualized learning. 
Online assessment solutions gave her ongoing indications 
of student progress. Interactive products, both basal and 
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supplementary^ augmented classroom lessons with tar- 
geted curricula focused on student outcomes. Reference 
materials at various levels^ both online and static^ enabled 
the students to conduct research matched to their abilities. 
And multimedia applications brought history to life. 

In an increasing number of schools, educators are using 
e-learning solutions to expand the course catalog or 
supplement the existing instructional material. Similar 
in many ways to their traditional counterparts, e- 
learning solutions are not difficult for schools to use. 

Most online courses are regular courses whose content 
follows a standard scope and sequence. Although the 
delivery methods are different from those used in tradi- 
tional courses, online courses are typically taught by 
certified teachers and follow specific curricula. Students 
read textbooks, write papers, take quizzes and exams, 
and participate in discussions — ^just like they do in tra- 
ditional courses. The major difference between a tradi- 
tional course and an online course is the physical and 
temporal separation between the class and the 
teacher — and often between members of the class. 

Instructional material in digital form offers resources in 
a new way, often with the added advantage of temporal 
currency or interactivity. For example, students using 
one e-learning application might be able to watch a 
graph be instantaneously redrawn when the parameters 
are changed. Students using another might be able to 
see the impact of changing an experimental variable or 
get up-to-date access to the latest vote in the Senate. 
Digital materials may also provide additional resources 
on demand, such as pronunciations, definitions, or 
background information. 

In the examples above, educators were able to resolve 
their problems because they had already integrated e- 
learning solutions into their education planning. So the 
online courses — in German and AP Calculus — ^were 
considered to be part of the school’s catalog of offer- 
ings. And the digital materials in the seventh-grade 
classroom were treated no differently than traditional 
textbooks, workbooks, or other supplementary materials. 

Thanks to investments at the federal, state, and local 
levels over the past 10 years, tremendous progress has 
been made to put the technology in place to support 
these solutions. During the past decade — according to 



Quality Education Data, Inc. — an estimated $59 billion 
has been invested in desktop computers, networking, 
Internet connectivity, and professional development. 
States have made deep investments in building the 
technology infrastructures of their schools. And com- 
munities nationwide have approved local bonds and 
levies to fund the hardware, software, connections, and 
training needed to level the technology playing field for 
their students. But despite these investments, and the 
benefits of e-learning, there is still more to do to ensure 
that children and teachers everywhere can take advan- 
tage of e-learning solutions. 

Many schools currently face obstacles as they try to use 
e-learning solutions. If they want to supplement their 
catalog with online courses or use locally delivered 
digital content, they face policy or funding barriers — or 
both. The schools may find that the online courses they 
want to offer do not meet current regulations regarding 
provider accreditation, teacher credentials, grading, or 
transcripts. They may not have the funds to pay for 
online courses. Or they may fear losing ADA-based 
funding when students “attend” courses online. In ad- 
dition, when they want to supplement classroom re- 
sources with digital content, they may discover that 
textbook dollars are not available. By recognizing these 
obstacles, we can address them — and thus move closer 
to creating e-learning-friendly environments. 

PART 2: Recognizing the obstacles 
associated with e-learning solutions 

For the most part, the obstacles to using e-learning so- 
lutions fall into the following three categories: 

• Policy (including issues of certification, teacher of 
record, credit, provider accreditation, and atten- 
dance) 

• Quality (including materials, instruction, and im- 
plementation of online courses as well as digital ma- 
terials used in the classroom) 

• Funding (including sources of money to purchase 
online courses and digital material) 

Online instruction, particularly in hard-to-staff subjects 
like foreign languages and advanced science and math 
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courses, is one way to ensure that students have access 
to the courses they need for college and life — regardless 
of teacher resources in their own school districts. How- 
ever, state lines often stand in the way of teachers pro- 
viding online instruction to students in states other than 
where they are certified — regardless of their academic 
qualifications. Other related obstacles involve regula- 
tions determining who can be the “teacher of record” 
for a course and what institution can grant credit. Defi- 
nitions of attendance can pose problems if students tak- 
ing online courses are off-site while “attending” class. 
Finally, because accreditation has traditionally been 
done on a regional basis, regional variations, such as in 
the number of annual school days, can pose problems. 

Questions about quality frequently arise (“How do we 
know it’s any good?”) when schools begin considering 
online courses or digital material. When discussing 
quality, it’s important to address the differences be- 
tween traditional and digital content as well as the de- 
livery mechanism for digital material. 

Funding is another problem area. In some instances, 
state educational regulations allow students to earn 
credit from an online course, but per-seat funding for- 
mulas cause the school district to lose corresponding 
funding. In addition, e-learning solutions, particularly 
supplemental digital content, have no consistent budget. 

PART 3; Removing the barriers to e- 
learning 

The responsibility for creating e-learning-friendly en- 
vironments lies with all of us involved in education in 
the United States — federal and state education agen- 
cies, state and local policy makers, and business. Here’s 
what we need to do as a start. 

• Establish a national research and development 
agenda for evaluating the ways that technology im- 
proves teaching and learning — and for creating a 
policy and funding environment that facilitates the 
use of technology for education. 

• Evaluate program regulations and change those that 
impede student access to expanded educational op- 
portunities. 



• Shift funding priorities and eliminate budgetary re- 
strictions that prevent the purchase of online courses 
or digital content. 

• Design online courses that meet nationally recog- 
nized content standards and staff the courses with 
fully qualified instructors. 

With regard to policy, quality, and funding, states that 
have e-learning-friendly environments have all — or 
most — of the following characteristics. 

Policy 

1. Teacher certification 

At present, each state has its own process and require- 
ments for obtaining a teaching credential. Organiza- 
tions such as the National Board for Professional 
Teaching Standards have established standards for na- 
tional board certification. To date, 35 states have ap- 
proved processes to recognize this kind of national 
certification. 

In an e-learning-friendly state: 

• All teacher certification meets the requirements of 
the Highly Qualified Teacher component of NCLB. 

• New teaching methodology standards, such as those 
of the National Association of Teacher Standards, 
have been developed for online teaching. 

• Additional requirements for teaching online are in- 
cluded in the certification process. 

• Teacher certification includes the requirement that 
all teachers understand e-learning solutions and be 
able to use them effectively. 

• State-level certification reciprocity is enacted. 

2. Teacher of record 

The teacher of record is the person responsible for as- 
signing student grades and authorizing course credit. 
Typically, the teacher of record is the certified teacher 
at the school who is teaching the course. 

In an e-learning-friendly state: 

• State and local policies allow the online teacher to be 
considered the official teacher of record. 
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• School districts that offer online courses as part of 
the standard course catalog accept the qualifications 
of the online teacher as the teacher of record. 

3. Credit 

The local school or district is traditionally the legal 
credit-granting institution for all students in atten- 
dance. Since online teachers are not members of the 
local faculty, some schools and local school districts 
are prevented by district or state policy from granting 
credit for an online course. In addition, attendance — 
as measured by seat time — has been the most common 
indicator of eligibility for credit. When students take 
online courses in non-school settings, such as libraries 
or at home, seat time may not be measurable. 

In an e-learning-friendly state: 

• State laws and policies allow local schools or districts 
to grant credit for any online course that is provided 
as a part of the school’s standard program. 

• School districts accept grades issued for an online 
course — on transcripts, for graduation requirements, 
and in calculating grade-point averages. 

• School districts accept successful completion of an 
online course (replacing seat time) for the purposes 
of granting credit. 

4. Provider accreditation 

Traditional schools are accredited by one of several re- 
gional accrediting agencies. Through organizations 
such as the Commission of International and Trans- 
Regional Accreditation (CITA), many large regional 
accrediting commissions maintain a common protocol, 
ensure standards, and conduct evaluations of distance 
learning providers. All protocol and standards for dis- 
tance education schools have been enacted and are now 
operational. 

In an e-learning-friendly state: 

• The regional accreditation agency has standards for 
evaluating providers of online courses. These stan- 
dards are comparable to those used to evaluate tradi- 
tional schools, but recognize the differences in 
delivery methods. 



• Evaluation standards for curriculum and instruction 
are equivalent to traditional school requirements. In 
addition, online course providers are evaluated on 
the use of available online technologies, instructional 
strategies, and online resources as well as on their 
appropriate use in enriching the student experience. 

• Public school districts use accredited providers. 

5. Attendance 

Accounting for student attendance is mandatory in tra- 
ditional schools. However, a student may “attend” an 
online course outside the school building or outside 
normal school hours. 

In an e-learning-friendly state: 

• An attendance policy for online courses recognizes 
that successful completion of a course is equivalent 
to attendance for the period of the course. 

• When a student takes an online course during an as- 
signed class period, attendance is taken as if the stu- 
dent were in a locally taught class. However when a 
student takes the course with a flexible schedule or 
location, appropriate successful progress through the 
course is used as a measure of attendance. 

Comment: Basic education funding is traditionally 
based on student attendance, with Average Daily At- 
tendance (ADA) or Full Time Equivalent (FEE) being 
the most common measure. In an e-learning-friendly 
state, schools maintain their basic education funding 
even if their students are taking an online courses in- 
stead of a traditional one. 

Quality of the student experience 

1. The quality of online courses 

To ensure the quality of online courses, three compo- 
nents must be evaluated: materials (curriculum and 
assessment), instruction, and implementation. To 
evaluate materials and instruction, educators apply ex- 
isting evaluation tools and processes in a new arena. 
Although the materials and teachers may be off-site, 
current processes can still work. 

Evaluating implementation requires a different ap- 
proach. In traditional courses, educators use classroom 
observation, quizzes, and tests to determine how well 
their students are learning. They also take note of at- 
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tendance and class participation. When students are at 
a distance, ongoing monitoring is more difficult, so 
schools have to be proactive in checking the effective- 
ness of online courses for their students. Selection of 
students, determination of how and where students 
participate, parental support, in-school support, and 
support from the online course providers are all issues 
that can impact a student’s chance for success. 

Many models can be used to provide an environment 
that is successful for students. In an e-learning-friendly 
state, some or all of the following critical factors are in 
place. 

Critical factors for curriculum and assessment materi- 
als in online courses 

• The online curriculum meets appropriate curricu- 
lum standards. 

• The content is appropriate for the grade level and 
age of students. 

• The use of technology enhances the curriculum. 

• The assessment content and methodology is 
appropriate to online courses. 

Critical factors for instruction in online courses 

• Teachers are certified and highly qualified to teach 
the course that they are teaching. 

• Teachers are proficient in teaching in an online envi- 
ronment. 

• Online teachers, like traditional teachers, are evalu- 
ated annually. 

• Online course providers provide qualification docu- 
mentation to their client schools or districts. 

• Appropriate metrics, such as for the teacher response 
time to students, are established. 

• Feedback to students is provided through appropri- 
ate communication vehicles. 

Critical factors for implementation of online courses 

• Students are selected to participate in online courses 
based on their potential for success: independence, 
motivation, academic preparation, and access to ap- 
propriate technology. 



• Every student has an on-site mentor to address prob- 
lems that may interfere with the educational process. 

• Mentors receive information to help them be effec- 
tive in supporting online students. 

• Mentors are responsible for the student’s learning 
space and technology. 

• Mentors communicate — about student progress and 
any problems that arise — ^with the course provider, 
the student, and the parent. 

• Every student in an online course that replaces a 
regular course is assigned a class period for the 
online course. 

• Online teachers hold regular office hours in which 
students can communicate with them. 

• Students, parents, and school personnel have access 
to current student reports. 

• Parents receive information to use in supporting 
their online students. 

• Schools and course providers have a workable feed- 
back mechanism. 

• Standards that support student success, such as re- 
sponse time for technical support and guidelines for 
the delivery of student work, are established by the 
online course provider. 

Comment: Schools can use existing online course 
evaluation tools to determine which online courses and 
implementation models best fits their students’ needs. 

2. The quality of digital instructional materials 

Educators already use an instructional materials selec- 
tion process to evaluate traditional materials, like text- 
books and supplementary print materials. Digital 
material should meet the same kind of educational 
standards with recognition of the differences in delivery 
mechanisms. 

In an e-learning-friendly state: 

• The instructional materials process and guidelines 
include standards that incorporate critical elements 
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for e-learning materials, such as the appropriate use 
of technology to support instruction. 

• Online course materials are evaluated using an in- 
structional materials process that is appropriate for 
digital materials. 

• Classroom e-learning solutions are evaluated with 
whatever instructional guidelines and processes are 
appropriate for their use. 

Funding 

1. Funding for online courses 

Local administrators have the legal responsibility for 
providing an education program that best serves students 
in their school. Today, public schools are funded on a 
per-student basis. Funding calculations are based on av- 
erage daily attendance or other site-based measures of 
student presence. Since online courses separate the 
physical presence of students and the educational proc- 
ess, they do not meet the conventional funding criteria. 

In most states today, online courses are not supported 
in the normal ADA/FTE funding process. The use of a 
financing methodology that fully funds students, re- 
gardless of whether they take in-class courses or online 
courses, will eliminate educators’ fear of losing their 
basic education funding. 

In an e-learning-friendly state: 

• Financing formulas allow schools to use basic educa- 
tion funding to pay for e-learning solutions, both 
online courses and digital content. 

• Title I regulations allow low-performing schools to use 
the Supplementary Education Services funds to pur- 
chase online courses and other e-learning solutions. 

• Policy makers have enacted funding formulas that 
provide fair reimbursement for e-learning solutions 
that are used to provide core instruction. 

Comment: Many states have elected to fund online 
learning through mechanisms such as state-funded vir- 
tual schools that have free or reduced tuition. In fact, 

1 5 states have done so, with an additional 7 states hav- 
ing cyber charter schools (Education Week, Technol- 
ogy Counts, 2004). Other funding mechanisms include 
extraordinary funding where the state compensates a 



virtual school for students who successfully complete an 
online course. However, that particular model removes 
funding from the local school that would have had the 
student as ADA/FTE. 

2. Funding for other e-learning solutions 

With current definitions of textbooks, most e-learning 
solutions (basal or supplementary) have to be purchased 
with supplementary materials funding or technology 
funding. Current resource allocation methods perpetu- 
ate the separation of traditional instructional materials 
and digital content. 

In an e-learning-friendly state: 

• The definition of ‘textbooks” has been expanded to 
include both online course materials and digital con- 
tent. 

• The textbook adoption process has been expanded to 
include digital content and online course materials — 
and provides appropriate guidelines and timelines for 
selection, purchase, and implementation. 

• Certain processes, like book-depository require- 
ments, have been amended to support the adoption 
of digital content and online courses materials. 

Call to action 

An e-learning-friendly environment is necessary for 
schools to provide a 2 E' Century education and can 
help them meet the goals of NCLB. To achieve such 
an environment, federal, state, and local educational 
agencies must create a long-term policy agenda that 
includes the normalized use of e-learning solutions. 
Policy makers must revisit the traditional standards and 
policies for delivering educational opportunities to stu- 
dents. They must also address the lack of flexibility in 
educational spending and the unyielding budget calen- 
dar that prevents local educators from providing the 
best educational opportunities for all students. 

The comments and scenarios in this paper reflect an 
urgent need. Today’s students, accustomed to revolu- 
tionary technology, are leaving the public schools to 
obtain faster, customized, and interactive online educa- 
tion from other sources. When local administrators 
have the power to make effective use of e-learning solu- 
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tions, public schools may again be able to provide the 
education these students are seeking. 

An e-learning-friendly state has solved policy, funding, 
and quality problems. If your state is not yet e-learning 
friendly, the ideas in this paper can serve as guidelines 
for change. 
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The Role of Educational Technology in Meeting 
the Promise of Supplemental Educational Services 

Steve Fleischman, Principal Research Scientist, American Institutes for Research, 
Director, Supplemental Educational Services Quality (SESQ) Center, www.tutorsforkids.org 



Introduction 

The supplemental educational services (SES) provi- 
sion of the No Child Left Behind (NCLB) Act of 2001 
establishes a significant new opportunity for families 
to choose, educators to provide, and eligible low- 
income children to receive effective, out-of-school 
academic support services. According to NCLB, 
supplemental educational services — essentially tutor- 
ing services provided^rr of charge to students — 
must be offered to eligible students from low- 
income families who attend Tide I schools that have 
been identified by their states as needing improve- 
ment — generally, by failing to make adequate yearly 
progress (AYP) for at least three years. ^ 

The SES provision expands options for parents of 
low-income children by enabling them to seek out 
these free, effective tutoring services. These ser- 
vices provide students extra help in academic sub- 



^ The services are to be paid for by districts, generally 
from their existing Title I, Part A allocations. Additional 
U.S. Department of Education information regarding 
these funding requirements is provided of pages 29-38 
of the Supplemental Educational Services Non-Regulatory 
Guidance (August 22, 2003), available at 
WWW. ed . gov/policy/ elsec/ guid/ suppsvcsguid. doc. The 
U.S. Department of Education’s definitions of key terms 
in SES (eligible child, eligible school, provider, supple- 
mental educational services) are provided in Appendix A 
to this paper. 



jects such as reading, language arts, and mathemat- 
ics. They are generally delivered outside the regu- 
lar school day — before or after school, on 
weekends, or in the summer. In addition to pro- 
viding expanded choice to parents, SES offers edu- 
cators another potentially effective way to boost 
the academic achievement of traditionally under- 
served youth. Einally, these mandated services 
provide policymakers an important accountability 
tool and support mechanism to make sure that all 
children receive the academic help they need. 

In 2004, this program will offer increased choice of 
supplemental services and educational opportunities 
to thousands of students in Title I schools through- 
out the country.^ Official estimates of the potential 
scope and impact of SES are not yet available. 
However, some observers estimate that SES ulti- 
mately may benefit as many as 1.5 million students 



^ Currently, national figures are not available to permit 
accurate estimates of the total number of identified 
schools and eligible students. This is because some states 
post lists of schools that have not made adequate yearly 
progress that do not clearly indicate which of these 
schools are required to provide SES. As of June 2004, 
the Supplemental Educational Services Quality (SESQ) 
Center (www.tutorsforkids.org) has identified 29 states 
that have posted clear lists of schools not making AYP, 
thereby allowing for an accurate count. In these states, 
more than 2,000 schools are required to provide SES in 
2004. 
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and create a marketplace for these services worth 
more than $2 billion annually.^ Although the long- 
term goal of the program is to reduce the need to 
provide SES, many more students may be eligible to 
receive the services in the short term. As of June 
2004, nationwide nearly 1,400 SES providers have 
been approved by the states. 

Similar to other large-scale education improvement 
initiatives, in the end, the committed and careful 
implementation of SES is likely to prove key to its 
success in meeting the needs of students. In this re- 
gard, state policymakers and administrators have a 
crucial role to play. This paper summarizes the SES 
provision, outlines key challenges to the successful 
implementation of the program, reviews the role 
that educational technology may play in improving 
the delivery of supplemental services, and offers 
ideas to turn the current implementation challenges 
into opportunities to help eligible children benefit 
from SES. 

SES: The Basics 

The SES provision engages a multiplicity of actors 
in its implementation: state policymakers and ad- 
ministrators, district and school leaders and staff, 
SES providers, families and parents, and ulti- 
mately, the students who receive the services. As 
an integral part of NCLB, the SES provision is put 
in motion after each state sets its own definition of 
adequate yearly progress (AYP), the minimum im- 
provement that districts and schools must reach 



^ Siobhan Gorman, a journalist who published a paper 
on the emerging SES marketplace in January 2004, cites 
these estimates. Steve Pines, executive director of the 
Education Industry Association, provides similar figures. 

Estimates of the number of state-approved providers 
vary, with the resulting total ranging from approxi- 
mately 1,400 to 1,800 providers. The SESQ Center de- 
rived the figure of 1 ,400 by counting each approved 
provider only once in each state. Thus, a national pro- 
vider of services with several approved local branch of- 
fices was counted only once in our tally. As a result, the 
maximum number of times that a provider can be ap- 
proved in the count is 52 all 50 states, the District of 
Columbia, and Puerto Rico. 



every year on state achievement tests and other 
measures (see Table 1). The law specifies that Ti- 
tle I schools that do not show AYP for three years 
or more must offer SES to students.^ Each state is 
charged with identifying schools that do not make 
AYP for at least three years. 

The states also must develop a process that results 
in a list of approved SES providers. States are 
supposed to identify providers that offer high- 
quality, research-based tutoring services in line 
with state standards. Each state creates its own 
application and establishes criteria for approving 
providers. At a minimum, states must require a 
program to demonstrate that it: 

• has a proven track record of raising student 
achievement in academic subjects (such as read- 
ing and math); 

• offers high-quality, research-based instruction 
focused on improving student academic 
achievement; 

• offers services that are in line with state stan- 
dards; 

• is financially sound; and 

• complies with federal, state, and local health, 
safety, and civil rights laws. 

States set their own application timelines but must 
give providers an opportunity to apply for state 
approval at least once a year. When reviewing 
provider applications, states can work to include 
input from district representatives, parents, 
teachers, and business and community partners. 
Once a state reviews applications, it must publish a 
list of approved providers, broken down by 



^ Eor farther details concerning the NCLB requirements 
regarding SES, see the U.S. Department of Education’s 
Supplemental Educational Services Non-Regulatory Guidance at 
WWW. ed . gov/ policy/ elsec/ guid/ suppsvcsguid . doc. 
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Table 1: NCLB School Improvement and Choice Options 



School 


Doesn’t make AYP 






Yr 1 








School 


Doesn’t make AYP 






Yr2 








School 


r'Yr of “school 


• 


Technical assistance 


Yr 3 


improvement” 


• 


Public school choice 


School 


2"“ Yr of “school 


• 


Technical assistance 


Yr4 


improvement” 


• 


Public school choice 






• 


SES 



Source: U.S. Department of Education, Office of Innovation and Improvement, Innovations in Education: Creating Strong Supplemental 
Educational Services Programs, Washington, D.C., 2004 



district. This list should at least indicate for each 
provider: 

• subject areas (such as reading or math) and 
grade levels covered; 

• provider’s track record in raising student 
achievement; 

• qualifications of the tutoring staff; 

• where and when the services are offered; 

• provider’s ability to meet the needs of students 
with disabilities or who are English language 
learners; and 

• provider’s contact information. 

A variety of public and private entities may offer 
the services, including non-profit groups; for- 
profit companies; local community programs; col- 
leges or universities; national organizations; faith- 
based groups; private and charter schools; and 
public schools and districts that have achieved 
AYP. Approved providers may offer these services 
through a variety of means. Many providers are 



offering “hands on” tutoring by trained instruc- 
tors. Others are providing Internet-based or com- 
puter-based instruction and academic support that 
students can access in a school, at a community 
center, or at home. The same state approval 
criteria apply to all applicants, including 
educational technology-based SES providers. 

In turn, each district with schools that fail to make 
AYP determines which students in its schools are 
eligible for the services.^ SES is available to low- 
income students in the school — generally, those 
students who are eligible for free or reduced-price 
lunch. 



^ If a Title I school does not make AYP for two years, it 
must offer all students in that school the opportunity to 
transfer to another public school that is not in need of 
improvement. Additional information regarding the 
“choice options” provided by NCLB are available on the 
U.S. Department of Education’s website at 
WWW. ed . gov/ about/ offices/list/ oh/ about/ choice .html. 
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Once the district determines which students are 
eligible, it notifies families that their children 
qualify for SES. The district must also provide in- 
formation about SES providers available locally, to 
help families make their selection. If families ask 
for assistance, a district must help them choose an 
SES provider. Once a family chooses a provider, 
the district enters into a contract with the provider 
and pays for tutoring services. If more eligible 
families request SES than existing funds can sup- 
port, districts must give priority to the lowest- 
achieving low-income students. After a family 
chooses, the provider, the school, and the district 
meet with the family to agree on performance 
goals for the child and a schedule for services. 

The state and district both play a significant qual- 
ity assurance role. The state has the overall re- 
sponsibility for the management of the program, 
including the approval and monitoring of SES 
providers. Under NCLB, states are to: 

• Monitor and report on the performance of the 
approved providers, 

• Provide a least annual opportunities for addi- 
tional providers to join the approved list, and 

• Remove from the state list any provider that has 
failed to help students improve their academic 
achievement for two or more years. 

Districts play an important role in ensuring the 
smooth fiinctioning of the program at the local level. 
They notify parents and facilitate choice, work with 
schools to coordinate services, enter into contracts 
with providers, and administer operational and finan- 
cial details of the program. States, in particular, have 
an opportunity to expand the number of effective 
SES providers, and assure quality implementation of 
the program. Suggestions on how they may do so are 
provided below. 



Potential Role of Educational Technology 
in SES 

Although there are many applications of educa- 
tional technology in the SES field, this paper con- 
centrates on describing the promise of providers 
who use either computer-based instruction (CBI) 
or online tutoring as a primary component of their 
programs. In practice, individual providers cur- 
rently offer CBI that may or may not be accompa- 
nied by a significant in-person tutoring or 
supervision component, and may be used as a core 
or supplemental instructional tool. Computer- 
based instruction may be used online or on-site, 
and to teach new content and skills or to provide 
additional review or practice. Similarly, there is 
variety in the delivery of online tutoring. Eor ex- 
ample, services may be offered live and 24 hours a 
day or at set times, and through a structured cur- 
riculum or more free-flowing student-tutor inter- 
action. Einally, even more “traditional” providers 
that rely on on-site tutors may use a significant 
technology component to supplement instruction 
via CBI, or to track and report on student pro- 
gress. 

Douglas A. Levin, an educational technology ex- 
pert at the American Institutes for Research, 
points that that for over thirty years, one of the 
most common uses of the computer in education 
has been to instruct and tutor students on chal- 
lenging academic content and skills. He explains 
that, “when used as a tutor, the computer presents 
material, evaluates responses, determines what to 
present next, and keeps records of progress (Kulik, 
1994). Students receiving tutoring with the aid of a 
high-quality computer tutor are provided with a 
structured path through a problem set, supported 
by some combination of visual, verbal, and sym- 
bolic cues that serve to improve student compre- 
hension of challenging material (Sivin-Kachala and 
Bialo, 2000).”' 



^ Personal communication to the author (June 6, 2004). 
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Table 2: SES Overview 





Step 1: Determine 
who is eligible 


Step 2: Identify 
providers 


Step 3: Review 
options 


Step 4: Make 
choices and begin 
services 


Step 5: Monitor 
and expand provider 
options 


States 

must 


■ Determine which 
schools need im- 
provement because 
they did not make 
“adequate yearly 
progress” (AYP) 

■ Identify schools 
that must provide 
SES because they 
have not made 
AYP for three 
years 


■ Encourage organiza- 
tions to apply to be 
SES providers 

■ Request applications 
from providers 

■ Review applications 
and approve providers 

■ Give districts a list of 
approved providers 

in their area 


■ Optional: Provide 
districts with sample 
materials for com- 
municating with eli- 
gible families about 
SES and contracting 
with providers 


■ Monitor the per- 
formance of ap- 
proved providers 
and report results 

■ Encourage more 
providers to apply 
for state approval 
at least once a 
year 


■ Monitor the perform- 
ance of approved 
providers and report 
results 

■ Allow more providers 
to apply for state ap- 
proval at least once a 
year 

■ Remove from the state 
list any provider that 
has not helped stu- 
dents improve their 
achievement for two 
or more years 


Districts 

must 


■ Identify students 
who are eligible 
for SES who at- 
tend schools that 
did not make AYP 
for three years 

■ Determine which 
students get prior- 
ity if demand ex- 
ceeds available 
funding 


■ Optional: Encourage 
local groups to apply 
to be SES providers 


■ Tell parents whether 
their child is eligible 
for SES 

■ Provide parents with 
clear, complete, 
consistent informa- 
tion about local pro- 
vider options 

■ Tell parents how to 
sign up for SES 


■ Flelp families 
pick a provider, 
if requested 

■ Enter into con- 
tracts with the 
providers that 
families choose 

■ Pay providers 

■ Work with each 
student’s provider, 
school and families 
to set goals for each 
student 


■ Provide the state 
with information to 
help monitor the per- 
formance of ap- 
proved providers 


Families 

can: 


■ Find out whether 
your child’s 
school made AYP 
by contacting his 
or her school, dis- 
trict, or state 




■ Find out whether 
your child is eligible 
for SES by contact- 
ing school, district or 
community group 

■ Review provider op- 
tions 

■ Select a provider 


■ Work with your 
chosen provider, 
your child’s 
school, and the 
district to set 
goals for your 
child 


■ Monitor your child’s 
progress with the 
provider and provide 
feedback to the pro- 
vider, school, dis- 
trict, and state 


Providers 

can: 


■ Determine 
whether your 
program has the 
capacity to serve 
students in 
regions where 
schools are 
required to 
provide SES 


■ Find and review the 
state SES provider 
application 

■ Complete and submit 
the application 


■ If approved, contact 
the school(s) and 
district(s) to estab- 
lish a relationship 
and provide staff 
with information for 
families about your 
program 


■ Provide clear in- 
formation to fami- 
lies and schools 
about your services 

■ Enter into contracts 
with the district(s) 
if students sign up 
for your program 

■ Work with your 
student’s family, 
school, and dis- 
trict to set goals 
for each student 
who signs up for 
your program 


■ Provide clear infor- 
mation to each stu- 
dent’s family, 
school, and district 
about his or her pro- 
gress in your pro- 
gram 



Source: SESQ Center 2004 
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Before examining the promise of SES, and tech- 
nology-based provision of these services, it is 
important to point out that as an innovative policy 
initiative, SES does not yet have scientifically based 
outcome evidence regarding its effectiveness. Nev- 
ertheless, prior research regarding the benefits that 
students similar to those targeted for support by 
this program can receive from tutoring and other 
out-of-school support services suggests that, if im- 
plemented well, SES will achieve its intended out- 
comes. Potential benefits established by prior 
research include: 

• Improved student achievement and work 
habits that are accomplished by building on 
learning that takes place during the school day. 

• Individual instruction focused on specific 
student needs that permits students who may 
not learn well in traditional classrooms to learn 
in different, perhaps more effective ways. 

• Reduced incidence of delinquent or risky 
behavior achieved by providing students a safe, 
supportive, and engaging environment outside 
of school. 

Specifically regarding the effectiveness of educa- 
tional technology. Levin points out that within the 
last five to seven years, innovations in computer- 
based tutoring have allowed specially trained edu- 
cators, supported by a suite of instructional and 
communication tools, to provide real-time tutor- 
ing to students near and far via the Internet. He 
argues that such approaches have the potential to 
offer additional instructional flexibility and per- 
sonalization in assisting students to master chal- 
lenging academic content. Levin points out that 
research on the effectiveness of computer-based 
tutoring for students has been conducted across a 
wide variety of subject areas, including read- 



^ For a review of this promising evidence, see 
Supplemental Educational Services Quality Center, 
Implementing Supplemental Educational Services: 
Opportunities and Challenges at 
www.tutorsforkids.org/ docs/PolicyBrief52 004.pdf 



ing/language arts, mathematics, and science. In 
these studies: 

• Students usually learn more in classes in which 
they receive computer-based instruction and 
have more positive attitudes toward the subject 
matter being studied (Eletcher, 2003; Kulik, 
1994; Kulik, 2003). 

• Students learn their lessons in less time with 
computer-based instruction (Eletcher, 2003; 
Kulik 1994). 

• The cost is reduced to provide such instruction 
versus comparably effective approaches 
(Eletcher, 2003; Levin, 1987; Solomon, 1999). 

Eurther, there is some promising research concern- 
ing the effectiveness of educational technology so- 
lutions in raising student achievement for 
disadvantaged youth, i.e., those most likely to be 
served by SES. However, Professor Steven M. 
Ross, a leading researcher on educational technol- 
ogy, points out that so far this evidence is still 
“mixed. Despite this. Dr. Ross contends that, 
“well designed educational technology programs 
can be highly effective.'' 

According to Ross, educational technology merely 
represents a potentially effective delivery vehicle for 
instruction and support. He suggests that to deter- 
mine whether an education technology-based SES 
program is potentially effective consumers will need 
to review the quality of the instruction and support 
offered, the curriculum that is used, and how well the 
program has met the needs of similar children in the 
past. Ross argues that all SES programs should be 



^ Personal interview (June 4, 2004). Dr. Ross is a profes- 
sor of educational psychology and the director of the 
Center for Research in Education Policy at the Univer- 
sity of Memphis. In addition, he is the longtime research 
editor of a leading academic journal on educational 
technology {Educational Technolog)! Research and Develop- 
ment), a member of editorial boards of several other 
educational research journals, and an advisor to a num- 
ber of states on SES implementation. 
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held to the same standards, regardless of whether 
they are technology-based or not. 

That said, Dr. Ross and others who have studied 
this emerging field note that education technology- 
based SES providers may offer significant benefits 
to policymakers and end users. In her recent re- 
view of the emerging SES marketplace of providers, 
journalist Siobhan Gorman points out that those 
who are providing online services “could fill an im- 
portant niche in serving students who don’t want to 
or can’t get to a site-based provider. Educational- 
technology solutions offer ways to address a number 
of implementation challenges — giving parents an- 
other significant choice of providers and educators 
another support to meet their students’ academic 
needs goals — because they: 

• Can be cost-effective to deliver. The actual 
amount that is provided for each student to re- 
ceive SES is set based on the district-level Title 
I per-pupil allocation, which generally ranges 
from $750 to $1,900. Providers in local market- 
places are free to determine whether to offer 
services based on these costs. In some locales 
where there are few students at an individual 
site (urban or suburban areas) or where students 
are hard to reach (remote or rural areas) SES 
providers who use distance learning technolo- 
gies may have a financial incentive based on 
lower operating costs, and an operational advan- 
tage in delivering services. Indeed, they may 
represent the only available providers. Thus, 
states that have many eligible students in “hard 
to reach” areas may have a special incentive to 
promote the entry of educational technology- 
based SES providers into the market. 



Additional information regarding the research evi- 
dence base for educational technology is available from 
the Center for Applied Research in Educational Tech- 
nology (http://caret.iste.org). 

“ Siobhan Gorman. The Invisible Hand of No Child Left 
Behind? American Enterprise Institute (AEI), January 15, 
2004. Available online at 

WWW. aei.org/ publications/ pubID .1973 0, filter, all/ 
pub_detail.asp. 



• May be more convenient and engaging to use 
than some “traditional” tutoring services. As 

with other applications of educational technol- 
ogy, SES services can be accessed from a variety 
of locations and at different times. This allows 
for great flexibility in the delivery and access of 
services, and may be particularly attractive to 
older students, who might otherwise not partici- 
pate. In this regard, it is useful to note that some 
online providers, such as SAdARTHINKING — 
which was already providing similar tutoring 
services live, online and 24 hours a day to col- 
lege students when the SES provision was 
launched — ^have entered the new marketplace be- 
cause of its perceived growth opportunities. In ad- 
dition, the novel approaches employed by a number 
of online and computer-based instructional services 
may offer students new and more engaging oppor- 
tunities to master academic material. 

• When well designed, technology-based pro- 
grams or online services can provide great 
consistency of instruction. Dr. Ross points out 
that because of their software design, and struc- 
tured interactions technology-based providers 
may be able to overcome some of the major 
challenges posed to “traditional” tutoring ser- 
vices, particularly those of recruiting, training, 
and assuring the availability of large numbers of 
high quality tutors. Providers who rely on soft- 
ware for primary instruction, or on online tutors 
who can interact with students in various loca- 
tions, may help to meet some of the early im- 
plementation challenges that SES faces. 

• Offer an opportunity to collect student per- 
formance data efficiently to track provider 
effectiveness. Many of the technology-based 
SES providers have automated their assessment, 
attendance, and reporting, providing an oppor- 
tunity to have accurate and ongoing evaluation 
and sharing of student progress. This can be a 
great advantage in terms of coordinating ser- 
vices with schools, reporting to parents, and 
providing the basis to monitor and judge pro- 
vider effectiveness. 
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Given the growing evidence base regarding their 
potential effectiveness, and the promise that 
education technology-based SES providers hold, 
state policymakers will need to find ways to 
facilitate their entry into the market, and to assure 
their quality once they begin to offer services. 

Current Status of Educational Technology 
Providers of SES 

It is difficult to determine the exact number and 
describe the activities of technology-based SES 
providers currently offering services. Estimates re- 
garding the number of active education technol- 
ogy-based providers of SES range from about 1 5 
to 50.^^ An accurate count is hard to derive since 
states do not currently report this information 
consistently, and it may not be clear from each 
provider’s description how much of their service is 
technology-based. Nevertheless, it is interesting to 
note that of the 1 5 most widely approved providers 
by the states (Table 3), at least six take an approach 
that employs a significant educational technology 
component (Table 4). Unfortunately, at present is 
impossible to estimate the number of students 
served by SES as a whole, or by any single provider. 

While well-known “traditional” tutoring providers 
such as Kaplan, Sylvan, Princeton Review, and Hunt- 
ington Learning Centers currently seem to dominate 
the market, and may incorporate technology into 
their services, it appears that others are leading the 
introduction of online or computer-based instruction 
and academic support. Some frequendy approved 
educational technology-based providers, such as 
Eailure Eree Reading, Kaplan, and Lightspan employ 
computer-based instruction on-site and online, often 



Siobhan Gorman estimates on the low end, and several 
industry representatives provided the author figures 
ranging from 20 to 50. However, all these are estimates 
based on limited evidence. All the experts consulted for 
this paper predicted a growth for this sector in the com- 
ing years. The SESQ Center is currently researching all 
approved providers to develop an accurate count. When 
complete, this information will be provided at 
WWW. tutorsforkids . org/ state . asp . 



supported by live instructors. Others, such as Bab- 
bage Net School serve as “virtual schools.” Einally, 
providers such as Brainfuse, Tutor.com, and 
SMARTHTNKTNG deliver a variety of online tutor- 
ing and academic support services. What unites these 
providers as a group is that they require access to 
computers and reliable online connections for the 
success of their services. 

According to industry representatives and other ob- 
servers, access — to students, schools, and necessary 
equipment and connectivity — ^is the key limiting fac- 
tor in the growth of education technology-based SES 
providers. As Christopher Gergen, one of 
SMARTHINKING’s founders and now a vice presi- 
dent at K12 puts it, “Getting approved is one-eighth 
of the battle.” The big challenge according to Ger- 
gen is getting into the schools, or to the students to 
deliver services. While in a sense, this is a challenge 
for all SES providers, Mark Schneiderman of the 
Software and Information Industry Association ob- 
serves that, “Perhaps a unique online SES issue is 
simply getting students access to an online computer 
with sufficient bandwidth.” Education technology- 
based SES providers face at least two major chal- 
lenges in terms of access: 



In preparation for this paper, the author communi- 
cated with the following education technology industry 
representatives: Christopher Gergen, Vice President for 
New Markets at K12, a leading provider of online cur- 
riculum and instructional tools to schools and 
homeschooling families (www.kl2.com) and a founding 
partner of SMARTHTKTNG (www.smarthinking.com); 
Steven Pines, Executive Director of the Education In- 
dustry Association (EIA), a professional network of more 
than 800 educators and education businesses dedicated 
to delivering and advancing the education of children 
and youth (www.educationindustry.org); and Mark 
Schneiderman, Director of Education Policy at the 
Software and Information Industry Association, the 
principal trade association of the software and digital 
content industries (www.siia.net). Policy statements re- 
garding SES implementation by EIA and SIIA addressed 
to leaders of the U.S. Department of Education were 
also reviewed. These documents are available on the 
websites of the respective organizations. 
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Table 3: Most Frequently Approved SES Providers 



Provider Name 


Website 


Number of States in 
which Approved 


Kaplan K12 Learning Services 


http://www.kaplan.com 


37 


Sylvan Learning Systems, Inc. 


http://www.sylvanlearning.com 


33 


Kumon Math and Reading Centers 


http://www.kumon.com 


32 


The Princeton Review, Inc. 


http://www.review.com 


27 


Huntington Learning Centers, Inc. 


http://www.huntingtonlearning.com 


27 


Lightspan, Inc. 


http://www.lightspan.com 


24 


HOSTS 


http://www.hosts.com 


23 


Club Z! In Home Tutoring Services 


http://www.clubztutoring.com 


22 


Babbage Net School 


http://www.babbagenetschool.com 


21 


Brainfuse Online Instruction 


http://www.brainfuse.com 


20 


1 CAN Learn Education Systems 


http://www.icanlearn.com 


20 


Tutor.com, Inc. 


http://www.tutor.com 


19 


EdSolutions, Inc. 


http://www.edsoultionsinc.com 


18 


Lindamood-Bell Learning Processes 


http://www.lindamoodbell.com 


17 


Smarthinking, Inc. 


http://www.smarthinking.com 


16 



Source: SESQ Center 2004 
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• Some districts and schools are reluctant to 
make their computer faciUties available to out- 
side providers. While districts have a host of le- 
gitimate concerns — ^particularly about security, 
wear and tear on equipment, and other costs asso- 
ciated with keeping computer labs open for extra 
hours — providers often complain that schools are 
either denying access to them outright or are 
charging exorbitant fees for the use of facilities. In 
some cases this lack of access to schools, currently 
the most convenient location for eligible students 
to receive services, seems to limit the benefits that 
might be derived fi*om SES. 



• Although education technology providers 
offer services that may be available “any- 
time, anywhere,” students may have limited 
access to those services outside of the school 
day. Currently, most online services require in- 
dividual, extended access to a computer, often 
equipped with sophisticated software and hard- 
ware. Students served by SES may not have ac- 
cess to this equipment or connectivity outside of 
school, or even at the schools some attend. In 
addition, given transportation issues, students 
living at a distance from school, including those 
living in rural or remote areas, may not be able 
to receive services at school because of schedul- 
ing constraints. 
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Table 4: Examples of Frequently Approved Technology-Based SES Providers 



Provider 


States in Which Approved 


Kaplan K12 Learning Services 
http://www.kaplan.com 
Approved in 37 states 


AL, AZ, AR, CA, CT, DE, DC, FL, GA, ID, IL, IN, lA, KY, MD, MA, Ml, MN, 
MS, NE, NV, NH, NJ, NY, ND, OH, OK, OR, PA, Rl, SD, TN, TX, VT, VA, WV, 
Wl 


Kaplan’s program helps students academically by providing live instruction from highly qualified instructors, or on- 
line instruction through multimedia Internet-based courses developed by educators. All of Kaplan’s programs in- 
clude research-based curriculum. 


Lightspan, Inc. 

httD://www.lightsoan.com 
Approved in 24 states 


AL, AR, CA, CO, DC, ID, IN, lA, KS, KY, MN, MS, MO, MT, NV, NH, NJ, NM, 
PA, SC, TN, TX, VA, Wl 


Lightspan, Inc. includes interactive software, school and home learning activities, teacher materials, on-site and 
technology-based professional development, and student assessment. It is research-based and aligns to individual 
state standards, textbooks, standardized tests, and professional association standards. 


Babbage Net School 
htto://www. babbagenetschool.com 
Approved in 21 states 


AL, CA, GA, ID, IL, IN, KY, Ml, MN, MO, MT, NJ, NY, ND, OH, OK, PA, SD, 
VA, WA, WV 


The Babbage Net School is a virtual school offering on-line, interactive courses in English, Math, Science, Social 
Studies, SAT, Foreign Language, Advanced Placement, Music, and Art. These courses are taught by certified 
teachers in a virtual classroom featuring interactive audio, synchronized web browsing and a shared whiteboard. 


Brainfuse Online Instruction 
htto://www. brainfuse.com 
Approved in 20 states 


AR, CA, CO, DE, FL, GA, IL, IN, KY, MA, MO, MT, NH, NJ, OH, OK, SD, TX, 
VT, Wl 


Brainfuse provides instant, unlimited access to tutors. With Brainfuse, students and teachers communicate by 
drawing on a virtual blackboard, typing instant messages and even speaking through online audio. 


Tutor.com, Inc. 

htto://www. tutor. com 
Approved in 19 states 


AL, AR, CA, DE, GA, IN, KY, Ml, MS, NV, NH, NJ, NM, NY, OH, OK, PA, VT, 
WV 


Tutor.com is an on-line service that offers students help with their homework. It is typically offered through local 
and school libraries and is available seven days a week. The students are connected with a tutor who assists by 
walking through the student’s homework. 


Smarthinking, Inc. 

htto://www.smarth inking.com 
Approved in 16 states 


AL, CA, CO, GA, HI, KS, KY, Ml, NV, NJ, OK, TX, VT, VA, WV, Wl 


Smarthinking, Inc. provides people, technology and training to help secondary and post-secondary institutions offer 
their students online tutoring and academic support. Schools, colleges, universities, government agencies, textbook 
publishers and other education providers work with Smarthinking to increase student achievement and enhance 
learning by connecting students to qualified educators anytime, from any Internet connection. 


Failure Free Reading 

htto://www.fai lurefree.com 
Approved in 13 states 


AL, AR, FL, GA, MA, MS, NJ, NC, OH, PA, TN, TX, VA 


Failure Free Reading’s materials include a combination of print, talking software, and teacher directed lessons. Software includes 
spelling, listening comprehension, story books, language development activities and more. Software also has Spanish resources 
for teaching English to Spanish speaking students. 



Source: SESQ Center 2004 



Note: These provider descriptions are drawn from each organization’s promotional literature. The table does not represent an endorsement 
of the selected programs, but rather demonstrates the range of services provided by widely adopted providers. Of the most frequently ap- 
proved providers, there are several that seem to use education technology in their program or method of delivery. The providers listed 
above use either computer-assisted instruction or online services as a core aspect of their tutoring and academic support. 
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Other challenges exist to the effective delivery of 
SES by education technology-based providers. 
While these providers have some potential advan- 
tage in reaching students once they have signed 
up, the fact that they may not have an “on the 
ground” presence in many communities makes 
their initial marketing job more difficult. Many 
“traditional” tutoring organizations — ^whether 
large “national” or small “mom and pop” opera- 
tions — ^will likely be better known in the commu- 
nity or in schools, and be able to build on 
established relationships. Marketing is expensive, 
and as one online provider commented, “It’s just 
not cost-effective to do outreach in a rural 
county. Thus, although online providers may 
offer one of the few alternatives in “hard to reach” 
areas, they may restrict their services in these be- 
cause of cost considerations. As well, these provid- 
ers may not be able to offer services to all the types 
of students eligible for services, because of lan- 
guage, physical, or other barriers. Finally, consum- 
ers of SES may be wary of technology-based 
services, since they may be less familiar with them 
and the services may seem unproven in meeting 
the needs of their children. This wariness may 
have some basis. As Dr. Steven Ross observes, 
more research needs to be done regarding the po- 
tential effectiveness of SES technology-based ser- 
vices, especially for children whose lack of success 
is in part due to the fact that they have not yet de- 
veloped the skills to be independent learners — a 
quality that may be key to the success of a number 
of technology-based programs. 

Prospects for SES; Creating Opportunities 

Despite the challenges faced by technology-based 
SES providers, they offer great promise in helping 
to assure that all eligible students are provided ef- 
fective tutoring and academic support services. 
Some SES implementation challenges are specific 
to technology-based providers (for example, 
greater access), but many other general challenges 
apply to them as well (greater participation, im- 



Gorman, The Invisible Hand of No Child Left Behind. 



proved provider quantity and quality). Thus, for 
SES to succeed, over the next few years, states, 
districts, and providers must work together to: 

• Increase access to SES, including educational 
technology-based services, in hard to reach ar- 
eas (whether urban, suburban, or rural). 

• Improve the information available to consumers 
regarding their SES options, including educa- 
tional technology-based services. 

• Expand the number of high quality educational 
technology-based SES providers. 

• Conduct additional research and evaluation of 
all providers, including educational technology- 
based SES providers. 

The following suggestions regarding how districts, 
states, and providers can cooperate to accomplish 
these goals are based on recent SES reports and 
policy briefs as well as on extensive interviews and 
conversations with program providers, researchers, 
state and district officials, and representatives of 
community organizations. 

A recent study, issued by the Office of Innovation 
and Improvement of the U.S. Department of Edu- 
cation, highlights some early examples of effective 
SES implementation practices drawn from five 
districts and points out the key role that they can 
play in assuring the program’s success. According 
to the report, for SES to succeed, districts must (a) 
embrace the spirit of SES, (b) build relationships 
with providers, (c) reach out to inform parents, 
and (d) set clear goals and track progress. Given 
the innovative and challenging nature of SES im- 
plementation, districts must early on move from a 
“compliance mode” to one that sets a positive 
tone, seeks out solutions, and plans for success. 
Key to this is seeing SES as a significant new op- 
portunity. As one district administrator in the 



U.S. Department of Education, Office of Innovation 
and Improvement. Creating Strong Supplemental Educa- 
tional Services Programs. Washington, DC, 2004. 

WWW. ed . gov/ admins/ comm/ suppsvcs/ sespro grams 
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study points out, “The turning point for us come 
when we began to see supplemental services as a 
great way to give extra support to the kids who 
needed the most help.” 

Furthermore, although states set many of the poli- 
cies that govern the operation of the program and 
the approval and monitoring of providers, districts 
have a significant opportunity to improve services 
by building strong working relationships with pro- 
viders. The on study provides a number of exam- 
ples and suggestions on how districts can: 

• Find out about, and create accurate profiles 
of providers who will work in the district. A 

number of districts in the study have provided 
families more extensive program information 
than is currently available through the state ap- 
proval procedure. This raises awareness among 
parents of their options and promotes better 
choices. It also allows district to get to know 
their providers and to establish a closer working 
relationship. 

• Work out reasonable provider access to facili- 
ties. This is the major complaint heard from pro- 
gram providers using technology-based 
approaches, although many other providers share 
the same concern. Some districts have not only 
encouraged the use of school site space, but also 
worked with community organizations to identify 
additional convenient sites to deliver services. 

• Use contracts that specify clear expectations 
for all parties and follow procedures that es- 
tablish a fair competitive environment. A 

contract that specifies responsibilities regarding 
attendance, billing, reporting, and performance 
is key to protecting families, students, schools 
and providers alike. The Oil study points out 
that “districts must strive to be impartial brokers 
in dealing with and communicating about SES 
providers.” Further, it cautions that if the “dis- 
trict itself is also an SES provider, it is especially 
important that it not inadvertently set up a sys- 
tem giving it an unfair competitive advantage 
over other providers.” 



• Increase communication and coordination 
between providers and schools. Key to the 

success of SES is a careful coordination of in- 
school and out-of-school instruction. For in- 
stance, several districts in the Oil study encour- 
age their teacher to share student data with pro- 
viders to support the development of individual 
SES student learning plans. 

• Expand the number and type of providers in 
the district. Although most of the work in ex- 
panding the provider base for SES will fall to 
states, districts should be alert to opportunities 
to encourage potential local providers — tutoring 
organizations, community and faith based or- 
ganizations, and others — to apply for state ap- 
proval. In “hard to reach” areas, the district may 
want to partner with these organizations to 
make sure that all eligible students have an op- 
portunity to receive services. 

A recent Supplemental Educational Services Qual- 
ity (SESQ) Center policy brief suggests additional 
steps that districts and states can take to improve 
SES implementation by sharing information and 
coordinating activities. For example, districts can 
help states by: 

• Offering input about the criteria and process 
for approving providers. While states are 
charged with the provider approval process, and 
need to play close attention to the SES provisions 
outlined in NCLB and the U.S. Department of 
Education’s Supplemental Educational Services Non- 
Regulatory Guidance, they should also seek out the 
advice of districts that are working hard to imple- 
ment the program. This will help to make sure 



The Supplemental Educational Services Quality (SESQ) 
Center — a project funded through a grant from the U.S. 
Department of Education — ^is working with a number of 
organizations to provide sample materials, networking op- 
portunities, training, and other assistance to help states, dis- 
tricts, and providers improve the implementation of the 
SES program. The policy paper is available at 
www.tutorsforkids.org/docs/policybrief6804final.pdf 
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that as far as is feasible district concerns are incor- 
porated into state actions. 

• Giving materials developed by the district to 
state officials to help other districts imple- 
ment SES. States can play an important role in 
identifying and sharing useful materials and 
practices of “early implementing” districts that 
can act as exemplars to guide other districts as 
they begin to provide SES. 

• Sharing feedback and data about provider 
effectiveness with state officials responsible 
for evaluating providers. Districts are likely to 
collect much of the data and some performance 
information needed to judge the quality and ef- 
fectiveness of SES providers. Therefore states 
and districts must work together closely to help 
states determine which providers should con- 
tinue to offer services, and which should be 
dropped from the state list. Additionally, this in- 
formation can help states develop technical as- 
sistance that promotes the entrance of more 
high quality providers into the market. 

Similarly, states can support districts by: 

• Coordinating the timing of notification 
about eligible schools and approved provid- 
ers to correspond with district timelines. 

Many eligible families are concerned that they 
do not receive enough time to learn about and 
select SES providers. States can help districts do 
their job better by providing “early warning” 
about which schools are likely to have to offer 
SES in the future, and by adopting a “rolling” 
application process for SES provider approval to 
assure a steady stream of providers in all eligible 
districts. 

• Developing sample materials for communi- 
cating with eligible families and contracting 
with providers. Many of these materials are 
likely to be applicable statewide. States can help 
assure effective and consistent implementation if 
they provide districts these supporting tools. 
States should consider expanding their guidance 
to families on how to make effective SES 
choices for their children, including how to se- 



lect from among education technology-based 
providers. 

• Providing guidance regarding contracting 
with technology-based providers. Districts 
may benefit from state guidance in the devel- 
opment of contracts that recognize the unique 
feature of online providers in terms of service 
delivery. The standard performance measures 
for service delivery — e.g., number, length, and 
frequency of sessions, and “attendance” — may 
not accurately describe the design or measure 
the effective delivery of online services. States 
should consider working with districts and 
online providers to determine fair, accurate 
measures that allow districts to assure effective 
and consistent delivery of services, while at the 
same time recognizing that technology-based 
services may need to be measured in differently. 

• Creating opportunities for officials from dif- 
ferent districts to exchange information and 
materials on SES. Sharing of information at 
events such as state SES “summits” for district 
leaders can build further support for the pro- 
gram and promote a valuable knowledge ex- 
change. The creation of regular emails or 
newsletters can support this sharing of best 
practices. 

• Providing technical assistance to potential 
district-based SES providers. States can en- 
courage the growth of choice at the district level 
by making the SES application process widely 
known, encouraging “non-traditional” appli- 
cants such as local community or faith based or- 
ganizations to apply, and providing local 
organizations support in the application and 
start-up process. 



Unfortunately, specific consumer information on 
choosing effective online providers does not exist. 
However, the SESQ Center provides guiding questions 
for parents to ask SES providers in general at 
WWW. tutorsforkids . org/ families/ infoforfamilies . asp . 
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• Developing an effective evaluation plan to 
monitor the quality of SES providers/^ This 
is perhaps the major challenge facing states that 
have implemented SES over the past two years. 
While states continue to explore the standards 
and evidence that will be applied in the provider 
re-approval process, they should consider the 
ways to develop a statewide technology-based 
evidence gathering and analysis system that will 
provide the accurate and comparable data 
needed to assure long-run program quality. As 
well, states need to develop effective approaches 
for measuring the impact of technology-based 
SES providers. 

Providers can also do their part to support states 

and district implementation. To accomplish this: 

• Providers or their industry associations 
should consider adopting “codes of conduct” 
or “practice standards” that will promote 
consumer confidence in their services. Eor 

example, the Education Industry Association, in 
cooperation with the Better Business Bureau, 
has developed standards for education services 
providers of tutoring services. 

• Industry associations should provide support 
and guidance to their members so that they 
can produce solid evidence that they are of- 
fering effective services. Eor example, the 
Software and Information Industry Association 
provides its member a Scientifically Based Re- 



The Council of Chief State School Officers provides 
some initial guidance on the state approval and evaluation 
process in its SEA Toolkit on Supplemental Services, available 
at WWW. ccsso . org/ content/ pdfs/ S SPT oolkit.pdf 

See www.educationindustry.org/newsletter/ 
newsletter.php?id=89. Several years ago, the American 
Association of School Administrators worked with a 
standards development panel that included education 
technology industry representatives to issue Standards for 
Web-Based Education Products and Services: Guidelines for 
K-12 Educators. The guide may be downloaded at 
www.aasa.org/issues_and_msights/technology/Ed.com_ 
brochure.pdf 



search: A Guide for Education Publishers & Devel- 
opers, in support of members’ efforts to improve 
the evaluation of their products. 

• Developers should work with districts to 
provide consumers accurate information on 

SES. Providers that team with districts and de- 
velop a solid working relationship can be highly 
effective in spreading the word about the value 
and availability of supplemental services. 

Einally, while this paper has concentrated on the 
“supply side” issue of improving the quantity and 
quality of technology-based SES providers, the 
“demand” side should not be overlooked. Nearly 
every report on early implementation of SES notes 
the generally low level of demand for these ser- 
vices from families. While this is the case, as Sio- 
bhan Gorman points out, as the SES program 
matures “the issues it will face are likely to be less 
about participation and more about access and 
quality.” As a result of state and district level ef- 
forts, and those of local and national community 
organizations and technical assistance providers, 
families are likely to receive more and better in- 
formation about available SES choices.^ ^ Never- 
theless, states, districts, and providers should 
continue to work together to make sure that fami- 
lies know what supplemental services are available 
and how to use them, have sufficient time to de- 
cide whether to take advantage of SES, and receive 
solid guidance so that they can make the best 
choices in the interests of their children. 

Conclusion 

In her report on early implementation, Siobhan 
Gorman observes that the “supplemental services 
program is perhaps the federal government’s larg- 
est free-market experiment going on in educa- 
tion.” The “marketplace” for SES is still being 



See www.siia.net/education. 

The SESQ Center provides links to a number 
of resources in support of SES at 
WWW. tutorsforkids . org/ resources . asp . 
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created. Many consumers are just learning about 
the services and how to use them. A variety of pro- 
viders are still testing the market, or just finding 
out about the opportunity to offer services. Finally, 
the market’s regulators and administrators are still 
working to develop a set of ground rules that as- 
sure a high quality supply of services. 

The emerging SES marketplace presents clear and 
promising prospects. SES expands access to high 
quality tutoring programs by giving low-income 
families the chance to choose free, quality services 
for their children. This program is an unprece- 
dented opportunity to benefit hundreds of thou- 
sands of students currently struggling academically. 
Providers have a significant opportunity to use edu- 
cational technology to improve the services that 
students receive. Finally, state policymakers and of- 



ficials can make key contributions to assure that the 
promise of SES is turned into effective academic 
support for our nation’s children. 

Steve Fleischman is a principal research scientist at the 
American Institutes for Research (AIR), and the director of 
AIR^s Supplemental Educational Services Quality (SESQ) 
Center. The SESQ Center (www.tutorsforkids.org) is a 
U.S. Department of Education funded national re- 
source center that helps low -income families take ad- 
vantage of the opportunities provided by the No Child 
Left Behind Act to provide their children free tutoring 
and extra academic help they may need. The views ex- 
pressed in this paper are the opinions of the author, and 
may not represent the position of the U.S. Department 
of Education, AIR, or the SESQ Center. Mr. Eleisch- 
man welcomes comments on this paper, which should he 
addressed to sfleischman@air.org. 
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Appendix A; Supplemental Educational Services Definitions 

Source: U.S. Department of Education. Supplemental Educational Services Non-Regulatory Guidance (Au- 
gust 22, 2003). p. 39. Available atwww.ed.gov/policy/elsec/guid/suppsvcsguid.pdf. 

Appendix A: Definitions 

Eligible Child: Eligible students are students from low-income families who attend Title I schools that are in 
their second year of school improvement, in corrective action, or in restructuring. Eligibility is thus deter- 
mined by whether a student is from a low-income family and the improvement status of the school the student 
attends [Section 1 1 16(e)(12)(A)]. 

Eligible School: A Title I school that must provide supplemental educational services. This includes (1) a Ti- 
tle I school that does not make adequate yearly progress by the end of the first full school year after having 
been identified as a school in need of improvement [Section 1 1 16(b)(5)]; (2) a Title I school that is in correc- 
tive action [Section 1 1 16(b)(7)]; and (3) a Title I school identified for restructuring [Section 1 1 16(b)(8)]. 

Provider: A provider of supplemental educational services may be a public or private (non-profit or for-profit) 
entity that meets the State’s criteria for approval. Potential providers include public schools (including charter 
schools), private schools, LEAs, educational service agencies, institutions of higher education, faith- and com- 
munity-based organizations, and private businesses. A provider (1) has a demonstrated record of effectiveness 
in increasing student academic achievement; (2) can document that its instructional strategies are of high qual- 
ity, based upon research, and designed to increase student academic achievement; (3) is capable of providing 
supplemental educational services that are consistent with the instructional program of the LEA and State aca- 
demic content standards, (4) is financially sound, and (5) abides by all applicable Federal, State, and local 
health, safety, and civil rights laws [Section 1 1 16(e)(12)(B) and Section 1116(e)(5)(C)]. 

Supplemental Educational Services: Supplemental educational services are additional academic instruction 
designed to increase the academic achievement of low-income students in low-performing schools. These ser- 
vices may include academic assistance such as tutoring, remediation and other educational interventions, pro- 
vided that such approaches are consistent with the content and instruction used by the local educational 
agency and are aligned with the State’s academic content standards. Supplemental educational services must be 
provided outside of the regular school day. Supplemental educational services must be high quality, research- 
based, and specifically designed to increase the academic achievement of eligible students. [Section 
1116(e)(12)(C)]. 
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How Can Virtual Schools Be a Vibrant Part of Meeting 
the Choice Provisions of the No Child Left Behind Act? 

Bryan C. Hassel, Michelle Godard Terrell, Public Impact 



Choice and the No Child Left Behind Act 

School Choice: Requirements and Benefits 

The accountability provisions of the No Child 
Left Behind Act of 2001 (NCLB) expand school 
choice opportunities for those attending public 
schools that are not meeting their state’s expecta- 
tions. Local school districts are required to provide 
children enrolled in low-performing Title I 
schools — identified as not making "adequate yearly 
progress (AYP)" for two or more consecutive 
years — the opportunity to attend an adequately 
performing public school while the original school 
is undergoing improvement. Choice must be of- 
fered to families in an eligible Title I school until 
the school is no longer identified for improvement.^ 

All students in a school identified for improvement 
must be given the opportunity to transfer to an- 
other public school, with priority given to the low- 
est-achieving children from low-income families. 
For example, if not all students can attend their 
first choice of schools, priority in assigning spaces 
would be allocated to the low-achieving low- 
income students. The types of educational choice 
options permitted by the legislation include trans- 
fers to higher-performing public schools within 
the district, charter schools, and virtual schools (as 
long as they are not Title I schools identified for 
improvement, corrective action or restructuring or 
identified by the state as persistently dangerous). If 
more than one eligible school is available, the LEA 
must offer more than one choice to eligible students. 



While NCLB’s requirements are one impetus for 
districts to offer more choices, the drive to increase 
options pre-dates the law. It rests on the accumulat- 
ing evidence that school choice can deliver important 
benefits for children - especially disadvantaged chil- 
dren. Wealthy and middle class families have long 
exercised school choice, either by sending their chil- 
dren to private schools or buying into communities 
with better public schools. But the choice option is 
now becoming available to low-income families stuck 
in schools in need of improvement. 

Research is beginning to show that school choice 
can be a very useful tool in improving educational 
opportunities for all, and particularly for disadvan- 
taged public school children. One line of research 
focuses on gains made in choice systems that allow 
private school enrollment as part of the choice 
menu (e.g. scholarships and vouchers). Taken to- 
gether, these studies suggest positive effects of 
choice for low-income African-American students.^ 
Studies of choice among pz/Mr school options have 
also shown benefits to children, including the chil- 
dren who “stay behind.”^ Since NCLB’s choice re- 
quirements are so new, there is less research 
specifically on its effects. But one recent study 
found that students in Chicago who transferred to 
higher-performing public schools saw much 
stronger achievement gains overall during the first 
year in their new schools than the year before."^ 
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Challenges to Meeting Requirements 

While most districts are complying to some degree 
with the law, many districts have not been able to 
meet the demand for transfers. In a recent report 
about the early implementation of the public 
school choice provisions in NCLB, researchers 
found that while parents express a strong interest 
in transferring their children to better-performing 
schools, many districts use the lack of school ca- 
pacity to deny families choices of some or all 
higher performing schools.^ 

Capacity issues continue to be a challenge for 
many districts. Though that exemption was elimi- 
nated in 2002, capacity issues continue to be a de- 
fense of many districts. Some districts simply deny 
NCLB transfers outright because of lack of capac- 
ity, while others put parents on notice that lack of 
capacity might cause their transfer requests to be 
denied.^ In many cases, these capacity and supply 
issues are real, particularly in rural districts, dis- 
tricts with limited transportation options, and dis- 
tricts with many schools in need of improvement 
and few high-performing ones. As a result, it is 
imperative to consider new ways to supply better 
options for children seeking transfers. 

Virtual Schools as a Solution to Capacity 
and Supply Challenges 

Online learning grew quickly over the past decade 
in universities and corporations, and more recently 
has become increasingly available to K-12 learners. 
While the exact number of virtual schools that are 
operating is unclear, a new brief from the Educa- 
tion Commission of the States about cyber schools 
presents the following statistics: 

• The Southern Regional Education Board esti- 
mates that over 100,000 students were enrolled in 
online courses during the 2002-03 school year. 

• Eifty-seven cyber charter schools were operating 
during the 2002-03 school year. 

• The Washington State Office of Public Instruc- 
tion found that 25% of Washington secondary 
schools had students enrolled in online courses 



during the 2001-02 school year and expected 
that number to triple by 2008.^ 

• Almost a third of school district leaders in a 2002 
survey predicted that more than one in five of 
their students would be receiving a "substantial 
portion" of their daily instruction online by 2005.^ 

The main differences between online learning and 
a traditional classroom are location and accessibil- 
ity. Online learning — simply defined as the use of 
multimedia technologies and the Internet for edu- 
cational content — can take on many forms. It can 
be purely online, with no face-to-face meetings, or 
provide blended learning, a combination of online 
and face-to-face learning. It can be synchronous 
(students working together and/or with instructors 
“live”) or asynchronous (students working largely 
on their own). Instruction can be provided by a 
subject matter expert, or a teacher guide, through 
collaborative exploration or largely through self- 
directed study. Instruction can also be facilitated 
by a “learning coach,” often the role played by lab 
attendants in virtual high school classes and par- 
ents in K-8 settings, who provides the face-to-face 
counterpart for a virtual teacher. 

One subset of online learning options is the “vir- 
tual school” or “cyber school.” While “online 
learning” could involve a single course or even a 
single lesson or project, a virtual school is a com- 
plete educational institution that delivers its in- 
struction primarily through online means. To 
fulfill the choice requirements of NCLB, a district 
must allow students to enroll in other schools. Sup- 
plementing their current school’s work with online 
enrichment, though potentially valuable, would 
not meet that requirement. As a result, the re- 
mainder of this paper focuses specifically on virtual 
schools as an approach to providing options under 
NCLB. Districts that truly suffer from lack of ca- 
pacity and supply may find that virtual schools are 
a viable solution for meeting the choice require- 
ments of NCLB. 

Forms of Public Virtual Schools 

Virtual schools serving K-12 public students gen- 
erally fall into one of the following categories: 
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• Schools Operated by regional agencies and 
consortia of educational entities, nonprofit 
and for-profit organizations 

At least eight virtual schools that serve multiple 
states are in operation across the nation. The 
Virtual High Schoof in Massachusetts allows 
6,000 students from around the country to par- 
ticipate in high school coursework in a self- 
paced environment. It offers full-year and se- 
mester-length courses, summer school courses 
for enrichment or credit recovery, and dual 
credit courses. Class size is limited to no more 
than 25 students and and “memberships” are of- 
fered to districts, collections of schools, individ- 
ual schools and individual students. Member 
schools offer one or more faculty members to 
join the network of teachers to provide instruc- 
tion, and in exchange for each teacher released 
by the school to teach a VHS course, the school 
is able to register 2 5 students per semester to 
choose from VHS’s catalog of courses. Each 
school must also identify a site coordinator who 
is trained to act as an advisor and administrative 
contact for VHS students in their school. 

• Schools operated by state education agencies 

At least 15 states are operating virtual schools. 
Typically, state-run virtual schools provide ad- 
vanced coursework or supplementary services to 
middle and high school students. An example is 
the Illinois Virtual High Schoof° which is de- 
signed to provide Illinois students enrolled in 
state public high schools increased equity and 
access to high-quality educational opportunities 
no matter where they live. The IVHS courses 
are aligned with the Illinois Learning Standards. 
Any Illinois high school student enrolled in a 
public, nonpublic or home school can partici- 
pate with approval from a local participating 
school rVHS Building Administrator. Students 
can take semester-length courses, summer term 
courses. Advanced Placement (AP) review 
courses, and ACT preparation courses. 

Nearly all virtual schools target middle and high 
school students. Florida, however, has embarked 
on a pilot program to see if full-time virtual 
schools are adequate for the younger population 



and currently is funding two virtual elementary 
schools. The Florida Connections Academy^ 
and the Florida Virtual Academy^^ each serve 
approximately 500 K-8 students under contract 
with the state department of education. For 
every student enrolled, the companies providing 
the educational program get a $4,800 voucher. 
Students get a loaned computer and free Inter- 
net access, and the schools send them supplies 
and books. They speak with a teacher over the 
telephone for progress reports, but parents or 
guardians serve as the primary instructors. 

• Schools operated by universities 

At least seven universities are providing online 
learning opportunities to K-12 students. The 
University of California Online College Prep 
Initiative (UCCP)^^ receives university and state 
funding to provide online college preparatory 
courses that are aligned to California content 
standards, and fulfill admission requirements to 
the University of California. The initiative of- 
fers AP and honors courses, plus tutoring and 
AP Exam Review to over 2,500 students at Cali- 
fornia high schools where college preparatory 
curricula are underdeveloped. One of the initia- 
tive’s primary goals is to provide opportunities 
to rural and low-income students to help them 
compete effectively for admission to leading 
universities. 

• Schools that are operated by local public 
school districts and other local education 
agencies 

At least 36 districts are operating virtual schools. 
These include the Evergreen Internet Academy 
(EIA)^"^ which has been in operation for five 
years as an alternative education opportunity in 
the Evergreen School District. For the first 
three years, teachers in the 7-12 grade school 
provided both online and traditional classroom 
instruction, but now full virtual instruction is 
provided, with courses offered to students be- 
yond the boundaries of the district. The school 
serves large numbers of students who were for- 
merly home schooled, as well as students in 
need of an alternative to the traditional brick- 
and-mortar program. Students can receive di- 
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plomas issued by the district or a Washington 
state diploma. Those outside the state of Wash- 
ington or enrolled in another school full-time 
can attend if they pay tuition. 

• Schools that receive a charter from a local 
district, state board, university or other 
sponsor 

The cyber charter school model of online learn- 
ing is the most prolific in the nation. At least 
ninety cyber charters are in operation, with Ari- 
zona, Ohio, and Pennsylvania leading other 
states in the number of virtual charter schools 
authorized. The 2T' Century Cyber Charter 
Schooh^ is chartered through the West Chester 
Area School District in Pennsylvania (but estab- 
lished through the cooperative efforts of the 
school districts in Bucks, Chester, Delaware and 
Montgomery counties). This school has per- 
formed particularly well, exceeding averages on 
state tests in more than half of their tested 
grades. 

Virtual Schools Are a Legitimate Option Under NCLB 

In February 2004, the U.S. Department of Educa- 
tion issued guidance specifically defining virtual 
schools as a legally acceptable way to create addi- 
tional capacity for students wishing to transfer. 
The Department views virtual education as a pow- 
erful technology innovation expanding opportuni- 
ties for “learning any time, any place” in support 
of the No Child Left Behind Act. As long as the 
virtual school is a public elementary or secondary 
school (as defined by state law) and has not been 
identified for school improvement, corrective ac- 
tion, or restructuring, a district may offer it to stu- 
dents eligible to transfer from schools in need of 
improvement. If a virtual school is not operated by 
the district, the legislation allows the district to en- 
ter into a cooperative agreement with the school 
so that its students can enroll. 

Possible Benefits of Online Learning 

There are numerous hypothetical benefits of 
online learning. Some have been researched well, 
while others need further exploration. Among the 
benefits most commonly touted by online educa- 
tion advocates are: 



• Enhanced communication among students 
and between students and teachers 

Because of the increased anonymity and the dif- 
ferent ways to communicate (discussion boards, 
instant messaging, emails, online presentations, 
etc), there may be increased communication be- 
tween class members and teachers. Students 
may feel more empowered to share their ideas 
and less afraid to pose questions. There may be 
a leveling of the playing field, as students inter- 
act with less regard to others gender, race, dress, 
and other factors. 

• Accommodation of different learning styles 

Materials can be presented in different ways (ex- 
ample: online notes and slides for the visual 
learner and teleconferencing for the auditory 
learner). Students with attention deficit disorder 
and anxious students can benefit from having 
the additional time to attend to and reflect on 
the subject matter before responding. Students 
may get more one-on-one attention and work in 
smaller groups than in the traditional classroom. 

• Unlimited, flexible, access to curriculum and 
instruction (any time, any place) 

Students who are learning off-site can download 
materials and work on the curricula at any time. 
Continual access to course documents lets stu- 
dents obtain materials at any time. 

• Frequent assessment 

Some online learning programs allow for daily as- 
sessment of how well as student has learned course 
content. Immediate feedback allows instructors to 
change their delivery of the content, as well as 
highlight weaknesses and strengths for students. 

• Increasing the supply of teachers 

Online learning allows students in different lo- 
cations to “share” top instructors, rather than 
limiting those instructors’ benefits to one place. 
In addition, teachers who have left the tradi- 
tional system may find working in an online 
learning situation to be particularly desirable 
due to scheduling, health issues, or work style. 

A 2001 survey of virtual schools found that access 

to an expanded curriculum was one of the most 
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frequently stated objectives of virtual school pro- 
grams/^ Virtual schools were found to have the ca- 
pability to extend equitable access to high quality 
education to students from high-need urban and 
rural districts, low-achieving students, and stu- 
dents with learning challenges. 

Other research has produced similar findings. A 
2001 cyber charter review prepared by KPMG 
Consulting for the Pennsylvania Department of 
Education suggested that virtual charter schools 
are able to provide an education to children who 
have been historically under-served by traditional 
school environments and programs. The nation’s 
first publicly-funded Virtual High School (origi- 
nally known as the Concord Virtual High School), 
a national consortium of high schools offering 
online courses taught and designed by cooperating 
teachers who are accredited in their respective 
states, has been seen as fostering independent 
learning and leveling the playing field for minori- 
ties, low-income students, and those in low- 
income areas. 

Harnessing Online Learning Options to Meet Choice 
Requirements of NCLB: Three Models 

How would a district actually offer its students vir- 
tual school options? We present three models of 
how this could work in practice and adequately 
meet the choice requirements of the legislation. 

Off-site Online Learning 

The first model is the more “traditional” off-site 
virtual school, where students access educational 
materials and instruction online from sites of their 
own choosing. This works particularly well for 
high school students who can work well without 
supervision. However, it does present problems for 
elementary-age school children who are from 
families where parents are working outside the 
home and cannot supervise their children, and for 
students who do not have ready access to the 
Internet or a quiet place to work. 

On-site Virtual School: Distinct “School Within A School” 
The second model creates a new school, which is 
housed within the old school building- a virtual 
school within a physical school. The old school 
could provide services such as the cafeteria, gym 



classes and other non-academic coursework. Stu- 
dents would continue to get on buses in their 
neighborhoods, eat lunch with their friends, and 
join their peers in art and music classes, etc., but 
core academic instruction would be provided 
online in a different room or structure located on 
the school site. This model is allowed by the 
NCLB legislation as long as it is a distinct school 
with its own governance structure.^° 

“Third Place” Virtual School 

In between those ideas is a type of online learning 
based not at a school or at home, but at an offsite 
facility in conjunction with a nonprofit organiza- 
tion, such as a community center. A teacher or 
administrator would be onsite to help monitor 
students; however, most instruction would be 
online. The energy and perhaps funding of the 
nonprofit organization could be tapped, possibly 
beyond just the provision of the facility. 

Any of these three models could serve as an allow- 
able option for students under No Child Left Be- 
hind. Integrating them into a district choice 
program, however, could present numerous chal- 
lenges for state, LEA, and federal policymakers. 

Challenges and Possible Solutions for 
Districts and States Using Virtual Schools 
to Fulfill NCLB Choice Requirements 

While online learning is an emerging approach for 
K-12 instruction, few states and districts have 
made the effort to develop and enforce policies 
that address the issues that are unique to virtual 
schools. States and districts interested in pursuing 
this option should first conduct a thorough analy- 
sis of existing policies to see if they support the 
implementation of virtual schools. If they do not, 
then new policies should be developed and 
adopted quickly. 

District and state policymakers and planners have 
numerous factors to consider in creating and oper- 
ating virtual schools, particularly under the frame- 
work of the NCLB legislation. These components 
include: 
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• supply and capacity; 

• funding; 

• housing; 

• enrollment boundaries; 

• teachers; and, 

• accountability. 

Supply and Capacity 

For a virtual school to be eligible to receive stu- 
dents under NCLB’s choice provisions, it must be 
a duly authorized public school under the laws and 
policies of the state and/or district. While the 
number of virtual public schools has grown in re- 
cent years, overall very few of them exist, espe- 
cially those that provide a full instructional 
program. In addition, many existing virtual pro- 
grams target secondary students. More elementary 
programs would be needed in order to meet the 
needs of younger students seeking transfers. Dis- 
tricts and states seeking to offer virtual school op- 
tions therefore will need to attend to “supply”: 
ensuring that there are enough spaces in virtual 
schools to meet the likely demand. 

Broadly speaking, there are two ways supply could 
arise. First, virtual schools could be created new. 
Second, pre-existing virtual schools could be au- 
thorized as legitimate public school options within 
the state or district. 

New Virtual Schools 

A district or state could create new virtual schools 
itself. Alternately, it could issue a Request-for- 
Proposals (RFP) inviting nonprofits, universities, 
groups of teachers, or other potential providers to 
submit applications to create new virtual schools. 
These could be charter schools, if the state’s char- 
ter law was hospitable to such schools, or they 
could operate under some kind of charter-like con- 
tract with the district or state. Either approach 
would require substantial investments on the part 
of the state or district. In the case of starting 
schools itself, the district or state would need to 
invest significant resources and develop the exper- 
tise necessary to create virtual schools. In the case 
of an RFP process, the district or state would need 
to develop criteria for selection and a review proc- 
ess. If these were already in place for a charter 



schools program, the challenge would be reduced, 
but the existing mechanisms might need to be 
adapted for the specific context of virtual schools. 

Pre-Existing Virtual Schools 

A district or state could also enact a process by 
which it authorizes existing virtual schools to be- 
come legitimate public school options. For exam- 
ple, a private virtual school could become a public 
school option if it contracted or chartered with a 
district or state, agreeing to abide by critical public 
school laws and regulations. Or, a public virtual 
school serving another district or state could be- 
come an authorized public school for a given juris- 
diction. As with new-school creation, this 
authorization process would require the establish- 
ment of an RFP, along with selection criteria and a 
review process. 

Recommendations to SEAs 

It is addressing the supply and capacity issue where 
states can take on the greatest leadership role. In 
particular: 

• States can ensure that the legal processes exist 
for the creation of new virtual schools and the 
authorization of existing virtual schools as eligi- 
ble public school options. This could involve 
enacting a charter school law, amending a char- 
ter law to ensure that it allows virtual schools, or 
enacting or amending policies that allow the 
state and districts to contract with outside enti- 
ties to manage public schools. 

• Districts could benefit by state education de- 
partments’ providing technical expertise in de- 
signing a program or providing guidance to 
district officials in choosing “ready made” pro- 
grams that would work well with local student 
populations and within their budget. State de- 
partment officials could provide assistance with 
grant-seeking for districts seeking start-up funds 
or ongoing operation funding. 

Recommendations to LEAs 

• Districts can begin by assessing the likely de- 
mand for virtual school spaces in their commu- 
nity. Such a needs assessment can then inform 
supply-creation efforts. 
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• A critical decision for the LEA is whether to 
provide virtual schooling directly, to rely on 
outside providers, or to utilize some combina- 
tion of in-house and outside supply. The key 
factors in this decision are the district’s expertise 
in online learning (or access to such expertise), 
the resources available to develop in-house capac- 
ity, and the viability of potential outside providers. 

Funding 

Determining who funds online learning programs 
and at what level is a key challenge for districts 
considering online learning program choices. One 
of the touted benefits of online learning is that it 
can be less expensive than providing instruction in 
“brick- and-mortar” structures. Virtual schools, for 
example, do not typically have the same costs in 
areas of transportation and facilities. The cost 
structure of virtual schooling would depend upon 
the particular model in use. All of the models 
would involve costs including computer and inter- 
net provision, instructor salaries and benefits, 
technology support, and per pupil licenses for any 
commercial products. An administrative staff, 
which could be headed by a lead teacher, a district 
or state official, or another designated individual, 
would need to be responsible for shaping policy, 
hiring/monitoring/firing teachers, ensuring that 
content meets local, state, and federal require- 
ments, making sure that delivery is high-quality, 
managing students (registering, scheduling, ensur- 
ing that they are participating, etc), ensuring that 
any technological problems are remedied quickly, 
and making themselves available (sometimes for 
extended hours) to deal with day-to-day issues. 

The “third place” model would also involve some 
facility expense. The onsite “school within a 
school” model could involve additional facility ex- 
pense, unless existing space could be reconfigured 
to accommodate the virtual program. The school- 
within-a-school would also incur additional costs, 
such as the resources (human and financial) required 
to provide food service, non-core classwork, etc. 

Some educators point out that start-up costs (devel- 
oping curriculum, learning the systems, and inte- 
grating the program) is the area where most virtual 
school planners can be overly optimistic about their 



capacity. Others point out that virtual learning does 
not necessarily decrease overall costs, rather ex- 
penses just are shifted to different areas. 

It is unclear how much funding is required to run a 
virtual school. A 2001 study of virtual schools sug- 
gested that state-run online learning costs an aver- 
age of $3,000 per student a year.^^ K12, a for-profit 
organization that provides a virtual curriculum to 
homeschoolers and cyber charters, however esti- 
mates that approximately $4,800 to $5,000 per 
student needs to be allocated to adequately support 
virtual schools. 

Typically, virtual schools run by states receive 
funding based on enrollments, but many states are 
still working through average daily attendance 
(“seat-time”) issues as they relate to virtual schools. 
State appropriations and state grants are a com- 
mon funding source for state-sanctioned, state- 
level virtual schools, and districts can also tap into 
such funds if available by state legislation. State, 
federal and foundation grants, and funding from 
districts receiving services, are also common. Ex- 
ternal funders often support virtual schools in or- 
der to promote equitable access to key curricula. 
“Barter” methods are used by some regional net- 
works or consortia, where members may trade a 
teacher-led course for student enrollments, and 
share consortium costs. 

Recommendation to SEAs and LEAs 

• Consider funding implications early on, includ- 
ing the level and funding mechanisms required 
by each of the three models presented. Per- 
pupil funding levels must reflect real costs of a 
quality non-classroom-based model. 

• Ensure that the costs of special education ser- 
vices to students who require them, including 
lEP modifications for the virtual environment 
and contracting expenses of any required face- 
to-face services are considered in the funding 
model. 

• Seek to identify as early as possible the most sus- 
tainable funding mechanisms for the program. 
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Housing 

A primary challenge to districts required to pro- 
vide another public school choice to students is 
where to place them physically. The models pre- 
sented in this paper offer three housing options: 
online instruction in the home (or other location 
arranged by the student’s family); online instruc- 
tion in an area set aside in the old school; or, 
online instruction in a third-party structure. 

The onsite online learning program allows dis- 
tricts to use existing space if available. Districts do 
not have to rethink transportation provision and 
other student services. The third place online 
learning program, though requiring negotiation 
with another organization, can access additional 
space if facilities are limited at the old school, and 
can bring the added benefit of partnership with a 
community organization. The offsite online learn- 
ing program can tap into “free” support from par- 
ents and eliminate all costs associated with a 
facility, but presents a major challenge to children 
who do not have parents or guardians at home 
during the school-day. 

Recommendations to SEAs and LEAs 
• Consider how best to deliver instruction to the 
specific population. Several questions must be 
addressed, including: If students are to receive 
instruction on their computers at home, how are 
elementary students to be cared for in families 
with both parents work outside the home? If 
students are to receive instruction at a “third 
place” facility, will a bus be provided to carry 
students there? Will the district provide virtual 
school students additional services, beyond core 
academic programs? If students go to school 
off-site or at a “third place,” will they be free to 
return to school for additional programs? 

Enrollment Boundaries 

Virtual schools often serve students from a wide 
geographic area, crossing districts, spanning across 
the state, and even multi-state areas. This can pre- 
sent confusion as to who is ultimately responsible 
for oversight and per-pupil payment flow with the 
expanded enrollment boundaries. 



Another issue that arises is that previously 
homeschooled students may want to enroll in the 
new virtual school. These students would not have 
been counted previously as students by the district 
and would not have received funding. If these stu- 
dents enroll, then the state needs to ensure that 
adequate funding is available to educate them. 

Recommendations to SEAs and LEAs 

• Determine enrollment boundaries for any vir- 
tual schools. If district-run and funded, would 
there be benefits in opening up the online learn- 
ing program to additional students from outside 
the district? Could the district earn revenue 
(from fees and tuition) from such outside en- 
rollment? 

• Develop policies, based in law, that clearly spell 
out who may be enrolled in the program and 
who is responsible for monitoring and funding 
the program. 

• For schools serving students in multiple states, 
clarify how individual state standards, account- 
ability provisions, and teaching quality require- 
ments will be handled. 

Teachers 

The delivery of the educational program online 
can be significantly different from teaching in a 
typical K-12 classroom. The instructor’s role 
switches from presenting content and providing 
in-person instruction, to engaging in communica- 
tion through a variety of instruments, ongoing as- 
sessment, and feedback. Critics of online learning 
programs for K-12 students rightly are concerned 
that competency and accountability of online fac- 
ulty can be worrisome. The NCLB requirement 
that, by 2005-06, all public school teachers be 
“highly qualified” can help to allay those worries, 
as these requirements would also affect online in- 
structors.^^ At the same time, these requirements 
can pose challenges of their own. State certifica- 
tion systems were built around the assumption of 
the teacher providing instruction to an identified 
group of students in a particular location. Do these 
adequately measure the competencies needed to 
teach in an online environment? Do they impose 
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restrictions that make little sense in such a setting 
(such as requirements that make it difficult to be- 
come certified in multiple states?) 

Another challenge is that districts, particularly small 
or rural ones, may find difficulty in accessing local 
teachers to provide online instruction in any of the 
three models we present. Virtual schools could, how- 
ever, make it easier for these districts to hire teachers 
from anywhere in the state and, if state law allows, 
from anywhere in the country or world. 

Recommendations to SEAs and LEAs 

• In addition, to ensuring that online instructors 
meet the NCLB requirements of being “highly- 
qualified,” LEAs and SEAs should consider im- 
plementing policies that require new online 
teachers to complete an approved professional 
development curriculum ensuring their compe- 
tency as online instructors prior to teaching stu- 
dents online and require experienced online 
teachers to demonstrate that they have the de- 
sign and implementation knowledge necessary 
to deliver quality instruction to students in the 
new school. Personnel policies should take into 
account the need for administrators of virtual 
schools to have a specific skill set and profes- 
sional development training which includes 
leading a teaching staff that may itself be com- 
pletely virtual. 

• SEAs could also inventory their teacher licensure 
requirements to ensure they do not impose re- 
strictions that would constrain virtual schools in 
ways not related to teaching quality. Eor example, 
states could reconsider policies that make it diffi- 
cult for a teacher certified in another state to 
teach local students, since virtual schools may 
want to employ out-of-state teachers. 

• Use the new instructional delivery model as a 
way to tap into labor pools that otherwise might 
not be available. Sources could include retired 
teachers and other teachers who are out of the 
system, possibly because they have young chil- 
dren, are pregnant, or live in locations that do 
not have job openings in their subject areas. 
Consider if trained paraprofessionals could pro- 



vide face-to-face supervision for students and 
assistance to virtual teachers in the “onsite” and 
“third place” models. 

Accountability 

Beyond meeting the requirements of NCLB, 

LEAs and SEAs will need to determine the ac- 
countability requirements of the virtual schools. 
Because the teachers, instructional delivery 
method, and housing of an online learning pro- 
gram may be completely different than the dis- 
trict’s traditional schools, traditional accountability 
standards may not work smoothly. Eor example, a 
system that relies on site visits and classroom ob- 
servations to gather data about schools would need 
to be adapted to the online context. A system of 
enrollment counts may need to be adjusted for the 
fact that a school’s students will not all be sitting in 
the same room in a certain day in October. An 
online learning charter school might be freed from 
many rules and regulations to which district 
schools would adhere, but the model that stays 
within the district may need to adhere to many of 
the same rules and regulations. How would com- 
pliance accountability work in this new setting? 
How would state testing work? Virtual schools of- 
ten have much more individual student perform- 
ance data than traditional schools - easy to 
document time on task, lesson completion, ongo- 
ing feedback, etc. Virtual schools, however, may 
need to arrange for face-to-face, proctored exam 
settings for state assessments, until the state system 
is more comfortable with online administration of 
standardized tests. 

Recommendations to SEAs and LEAs 

• Develop and implement a contract that spells 
out all expected educational, operational, and fi- 
nancial expectations, and provides a specific 
process and consequence for failing to meet the 
agreed upon goals. 

• Consider identifying additional assessment methods 
or adapting existing methods so that they are ap- 
propriate to the online learning setting. 

• SEAs may choose to take on a supporting role 
and developing a list of goals so that all online 
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learning in the state is held to the same level of 
scrutiny. 

Recommendations for Federal 
Policymakers 

The federal government could also play important 

roles in making virtual schooling work as an 

NCLB choice option, including: 

• Using non-regulatory guidance to describe what 
counts as a “virtual school” (for purposes of 
NCLB choice). This definition is especially im- 
portant in the onsite online model, in which the 
district is offering a virtual school-within-a- 
school. Without clear guidance about require- 
ments for separate faculty and separate govern- 
ance of the virtual school, this model could 
easily degenerate into something other than a 
real choice for families. For example, giving stu- 
dents the chance to spend an hour a day in a 
computer lab working unsupervised on Internet 
research would not constitute a “virtual school.” 
But in less extreme cases, the line would be 
more difficult to draw. Federal guidance would 
help.^"^ 

• Provide start-up funds for new virtual schools. 
These new online learning programs may ex- 
perience many of the same challenges experi- 
enced by start-up charter schools. The federal 
government should consider developing start-up 
grants for online learning programs that helps 
the school to plan and launch its inaugural year. 
For virtual charter schools, federal public char- 
ter school funds are already available for this 
purpose. Federal officials could review other ex- 
isting federal programs to determine whether 
starting up virtual schools would be an eligible 
use of these funds. 

• Serve as an information-clearinghouse on solu- 
tions to the challenges discussed above. As states 
and districts develop solutions to problems such 
as those related to supply, funding, housing, 
teaching quality, and accountability, the federal 
government could play an important role in 
gathering and disseminating promising prac- 



tices, as it has already with district choice and 
supplemental services programs more generally. 

Conclusion 

Virtual schools are an acceptable, legal option for 
districts and states seeking to increase their capac- 
ity to meet the choice requirements of the No 
Child Left Behind Act. Research demonstrates 
that they can offer high-quality instruction to K- 
12 learners regardless of location, family income, 
background, or learning differences. While this 
research is too new and tentative to warrant any 
kind of large-scale shift to virtual schooling, it is 
strong enough to suggest that districts and states 
should be experimenting to a much greater degree 
with virtual schools. 

If districts and states decide to use virtual schools 
to meet NCLB’s choice requirements, however, 
they need to address a panoply of issues related to 
the implementation of this option. Ideally, virtual 
schools would be part of a coherent districtwide or 
statewide choice program. According to a U.S. 
Department of Education’s publication, promising 
practices in district choice programs include: com- 
petent leaders and staff, a true partnership with 
parents and the community, the perspective that 
accountability and competition are positive, and a 
strong strategy with appropriate resource alloca- 
tion, strong infrastructure, and proactive commu- 
nication.^^ 
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Key Terms 

Asynchronous communication: Communication in 
which students and instructors interact at various 
times (examples include e-mail, threaded online dis- 
cussions, and homework message boards). 

Brick-and-mortar school: An educational or- 
ganization that enrolls students primarily in class- 
room-based courses located in a school facility. 

Online learning: Instruction and content deliv- 
ered primarily via the Internet. 

Online learning program: An educational or- 
ganization that develops and offers online instruc- 
tion and content. An online learning program may 
be a virtual school, or it may provide only supple- 
mentary services for students enrolled in brick- 
and-mortar schools or virtual schools. 

Supplemental online program: A part-time 
online learning program that offers courses or 
other learning opportunities to students who are 
otherwise enrolled in brick-and-mortar schools or 
virtual schools; credit for successful completion of 
these learning opportunities is awarded by the 
brick-and-mortar school or virtual school in which 
the student is enrolled. 

Synchronous communication: Communication in 
which students and instructors interact at the same 
time (via instant message, telephone calls, face-to- 
face meetings, chatrooms, videoconferencing). 

"Virtual school or cyber school: An online learning 
program in which students enroll and earn credit to- 
wards academic advancement (or graduation) based on 
successful completion of the courses provided by the 
school. Credit for successful completion of these leam- 
ing opportunities is awarded by the virtual school. 

Online Resources 

Any TimCy Any Place^ Any Path, Any Pace: 
Taking the Lead on Online Learning Policy 
National Association of State Boards of Education, 
October 2001 

www.nasbe . org/Educational_Issues/Reports/ 
e_learning.pdf 



Beyond Brick and Mortar: Cyber Charters 
Revolutionizing Education, 

Center for Education Reform, January 2002. 

WWW. edr eform . com/ index, cfm ? fuse Action= 
document&documentID= 1 00 1 

Choosing Better Schools: A Report on Student 
Transfers Under the No Child Left Behind Act 
Citizens’ Commission on Civil Rights, May 2004 
www.cccr.org/ChoosingBetterSchools.pdf 

Cyber and Home School Charter Schools: How 
States are Defining New Forms of Public Schooling 
National Center for the Study of Privatization 
in Education 

www.ncspe.org/publications_files/ Cyber% 20and 
% 2 OHome % 2 0 Charters .pdf 

Distance Learning for K-12 Students 
Distance Learning Resource Network 
www.dlrn.org/kl 2/index.html 

Electronic School 
www.electronic-school.com/ 

E-School News 
WWW. eschoolnews . org/ 

Trends and Issues, A Study of Virtual Schools in 
the United States 

Distance Learning Resource Network and The 
Center for the Application of Information Tech- 
nologies, 2001 

www.wested.org/online_pubs/virtualschools.pdf 

Virtual Learning and Charter Schools: Issues and 
Potential Impact 

Southern Regional Education Board 

www.sreb.org/programs/EdTech/pubs/PDE/ 

Virtual_Learn_Charter_School.pdf 

Virtual School List 

Distance Learning Resource Network 

www.dlrn.org/kl 2/virtual_list.html 
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Meeting the Need for High Quality Teachers: e-Learning Solutions 

Glenn M. Kleiman', Education Development Center, Inc. (EDO 



Educators, researchers, policymakers, and parents all 
agree that high quality teachers are the most important 
factor in a child’s education. Rice (2003) reports that 
the relevant research shows the following: 

• the single most important factor affecting student 
achievement is teachers, and the effects of teachers 
on student achievement are both additive and 
cumulative; 

• an analysis of 400,000 students in 3,000 schools con- 
cluded that while school quality is an important de- 
terminant of student achievement, the most 
important predictor is teacher quality; 

• the estimated difference between having a good 
teacher and having a bad teacher can exceed one 
grade-level equivalent in annual achievement 
growth; 

• measures of teacher preparation and certification are 
by far the strongest correlates of student achieve- 
ment in reading and mathematics, both before and 
after controlling for student poverty and language 
status; 

• lower achieving students are the most likely to bene- 
fit from increases in teacher effectiveness. 

Our nation’s investment in teachers is enormous; there 
are more than 2.7 million full-time teachers employed 
in K-12 public and private schools (NCES, 2004), and 
the cost of teachers is the largest component - esti- 
mated at about 50% - of the more than $400 billion 
our country will spend each year on public K-12 educa- 
tion (NCES, 2003). 



No Child Left Behind and MOUSSE Teacher 
Quality Requirements 

In response to the critical need for a high quality 
teacher in every classroom, the No Child Left Behind 
legislation requires that all teachers of core academic 
subjects" be highly qualified by the end of the 2005- 
2006 school year. The NCLB teacher quality require- 
ments are written in broad terms, so that each state can 
develop its own specific teacher quality criteria and 
measures. Specifically, the NCLB definition of “highly 
qualified” requires that the teacher: 

1 . Has obtained full state certification or passed the 
state teacher licensing examination and holds a li- 
cense to teach in the state; 

2. Holds a minimum of a bachelor’s degree; 

3 . Has demonstrated subject matter competency in 
each of the academic subjects he or she teaches. 

The specific requirements differ for new and for ex- 
perienced teachers, and for elementary, middle, and 
high school teachers. New elementary school teachers 
must demonstrate competency by passing a rigorous 
state test on subject knowledge and teaching skills in 
reading and language arts, writing, math, and other ar- 
eas of the elementary curriculum. New middle and sec- 
ondary school teachers need to demonstrate subject 
knowledge either by completing an academic degree or 
its equivalent or by passing a rigorous state test in the 
specific subject area or areas they teach. Experienced 
teachers can meet the same standards as new teachers 
or meet a High Objective Uniform State Standard of 
Evaluation (HOUSSE), which each state has the option 
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to develop. HOUSSE enables practicing teachers to 
demonstrate knowledge of their subject area without 
necessarily taking a test or participating in further for- 
mal study. 

The Education Commission of the States (ECS) 

(www. ecs . org/ clearinghouse/ 49/ 6 8/496 8 . doc) reports 
that HOUSSE systems developed by states use the fol- 
lowing types of criteria: 

• Point systems that provide teachers with credit for 
activities related to the subject taught, such as 
coursework, professional development, service on 
curriculum development committees, professional 
presentations, and publications. 

• Professional development activities, often allowing 
teachers to develop plans that will enable them to 
meet the competency standard by 2006. 

• Performance evaluations, including observations and 
reviews by peers, a panel, or a supervisor. 

• Portfolio assessments that provide collections of evi- 
dence from the teacher’s practice, such as lesson 
plans, student work, and classroom artifacts, that 
demonstrate competency in the subject taught. 

• Student achievement data that measures the 
teacher’s impact on their students’ achievement. 

ECS also provides summaries of the HOUSSE 
provisions developed by each state 
(www. ecs . org/ ecsmain. asp ? page=/html/ educationissues/ 
teachingquality/housse/houssedb_intro . asp) . 

The teacher quality standards drive a need for substan- 
tial professional development to ensure that current 
teachers have opportunities to meet the standards. In 
response, NCLB provides substantial funding to the 
states for professional development programs. Title I 
Part A supports professional development for teachers 
of disadvantaged students; Title I Part B, the Reading 
Eirst and Early Reading Eirst programs, provides fund- 
ing to each state for programs that prepare teachers to 
implement effective reading instruction. Title II Part A 
provides nearly $3 billion a year for the Improving 
Teacher Quality State Grants program. Other parts of 
Title II support programs to enhance teacher subject- 
matter knowledge in mathematics and science; district 
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efforts to recruit, train, and retain individuals from 
other careers to become teachers in high-need schools; 
and programs that prepare teachers to use technology 
to enhance education. Title III supports professional 
development programs for teachers of English lan- 
guage learners; Title V supports programs to increase 
the highly qualified teacher population in order to re- 
duce class size. Other NCLB Titles and other federal 
programs, such as the Individuals with Disabilities 
Education Act (IDEA), provide additional funding for 
professional development. 

Teacher Quality Challenges for States, 

Districts, Schools, and Teachers 

While NCLB sets the teacher quality requirement and 
provides substantial funding, meeting these require- 
ments presents major challenges for states, districts, 
and schools, as well as for individual teachers. These 
challenges include the following: 

1 . In many areas, there are large numbers of under- 
qualified teachers who do not meet the state certifi- 
cation requirements but are teaching with emer- 
gency permits or credential waivers, since adequate 
numbers of credentialed teachers have not been 
available. Eor example, in 2002-2003, 7.2% of the 
K-12 teachers in California schools - more than 

2 1,500 teachers - were teaching with emergency 
permits or credential waivers (CCTC, 2004, p. 2) 

2 . There is an inequitable distribution of under- 
qualified teachers. Eor example, the 2003 California 
study on The Status of the Teaching Profession finds 
that students in the lowest-achieving schools, meas- 
ured by the state’s Academic Performance Index, 
are 4.5 times more likely to face under-prepared 
teachers than students in the highest-achieving 
schools (CETL, 2003). 

3. There is a high turnover rate of teachers, with about 
a third leaving the profession within their first three 
years of teaching and almost half leaving within 
their first five years. In low-income urban schools, 
the turnover rate is even higher than the national 
average (NCTAE, 2003). 
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4. There is a shortage of qualified teachers in particu- 
lar subject areas, such as mathematics, the sciences, 
foreign languages, and special education (Darling- 
Hamond, 2000, p. 7). 

5. Many pre-service teacher preparation programs are 
inadequately preparing graduates to meet the state 
teacher quality standards (ACE, 2004; US DoE, 
2003). 

6. There are specific issues at the middle school level, 
in which many current teachers have elementary 
credentials and lack adequate expertise in the sub- 
ject area they teach. 

7. Small rural schools often have teachers who teach 
multiple subjects and therefore must demonstrate 
subject matter expertise in each one. (Recent NCLB 
guidelines provide additional time for eligible 
teachers who meet high quality standards in one 
subject area to do so in other areas they also teach. 

8. About 700,000 teachers are projected to retire over 
the next 10 years (NCTAE, 2003), and some dis- 
tricts have reported teachers deciding to retire ear- 
lier than they had previously planned rather than 
meet the new teacher quality requirements. 

9. There is a shortage, in many places, of professional 
development opportunities that directly address what 
each teacher needs in order to reach the teacher 
quality standard. This is particularly true in rural ar- 
eas, but teachers throughout the country may be 
challenged to find high-quality professional devel- 
opment that fits their specific needs and is available 
when and where they can participate (ACME, 2002). 

Clearly, solutions to this myriad of challenges requires 
addressing the full range of systemic issues that under- 
lie the problems of teacher recruitment, training, and 
retention. However, e-learning can contribute to ad- 
dressing each challenge by enhancing the preparation 
of new teachers, providing high quality and readily ac- 
cessible professional development opportunities for ac- 
tive teachers, and making the teaching profession more 
attractive (e.g., by providing online resources for teach- 
ers and new connections to colleagues and mentors) to 
help address the teacher recruitment and retention 
problems. This paper considers the potential of e- 
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learning as a powerful new tool to help address the 
teacher quality challenges. 

Principles of Effective Professional 
Development 

In order to be effective, e-learning for teachers must 
reflect the principles of effective professional develop- 
ment. Research has led to agreement on a number of 
key principles of successful professional development 
practices for K-12 educators. In a summary of these 
principles. Sparks and Hirsh (1997) describe a “para- 
digm shift” in staff development, away from one-day 
in-service presentations to professional development as 
an integral, ongoing part of teachers' lives. Major re- 
search studies and syntheses by Shulman (1987), Stigler 
and Stevenson (1991), Darling-Hammond and 
McLaughlin (1995), Sparks and Hirsch (1997), Ball and 
Cohen (1999), National Eoundation for the Improve- 
ment of Education (1996), National Staff Development 
Council (2001a), Borasi and Eonzi (2002), and others 
consistently agree that professional development is 
more effective when it: 

• fosters a deepening of subject-matter knowledge, a 
greater understanding of learning, and a greater ap- 
preciation of students' needs; 

• centers around the critical activities of teaching and 
learning — planning lessons, evaluating student work, 
developing curriculum, improving classroom prac- 
tices and increasing student learning — rather than on 
abstractions and generalities; 

• builds on investigations of practice through cases 
that involve specific problems of practice, questions, 
analysis, reflection, and substantial professional dis- 
course; 

• values and cultivates a culture of collegiality, involv- 
ing knowledge and experience sharing among educa- 
tors; and, 

• is sustained, intensive, and continuously woven into 
the everyday fabric of the teaching profession, 
through modeling, coaching, and collaborations. 
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These principles are reflected in the NCLB legislation, 
which specifies that high quality professional develop- 
ment includes, but is not limited to, activities that: 

• Improve and increase teachers’ knowledge of aca- 
demic subjects; 

• Are integral to broad school-wide and district-wide 
educational improvement plans; 

• Give teachers and principals the knowledge and skills 
to help students meet challenging state academic 
standards; 

• Improve classroom management skills; 

• Are sustained, intensive and classroom-focused; are 
not one-day or short-term workshops; 

• Advance teacher understanding of effective instruc- 
tional strategies that are supported by scientifically 
based research; 

• Are developed with extensive participation of teach- 
ers, principals, parents and administrators (US DoE, 
2004 ). 

The Potential of e-Learning 

While there is general agreement about the core prin- 
ciples that underlie successful professional development 
programs, these principles can be difficult to imple- 
ment in actual practice because the required expertise, 
time, funding, and culture of collaboration may not be 
readily available at a school or district. In addition, the 
high quality standards drive a need for subject area and 
grade-level specific professional development, and the 
specific program a teacher needs may not be available 
locally, especially in rural areas and for teachers of spe- 
cialized courses. This situation, combined with the 
widespread access to the Internet in K-12 schools, 
teachers’ homes, libraries, and other community cen- 
ters, has led to the increased use of e-learning as a vehi- 
cle for delivering professional development targeted to 
specific teachers’ needs. 

There are many forms of e-learning courses. The ex- 
ample programs described below primarily use an online 
professional learning community approach, in which a co- 
hort of educators participates in a series of learning ac- 
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tivities, exchanging ideas with others in the cohort as 
well as with the instructor. This approach generally 
uses readily available web-based technologies and asyn- 
chronous discussions, so teachers can participate on 
their own schedules from any location with Internet 
access. Courses using a learning community approach 
often include classroom or school-based activities in 
which teachers are asked to implement a sample lesson, 
prepare lesson plans, assess students’ work, or visit 
other classrooms, and then discuss these activities 
online or incorporate their work into their projects. 

Other e-learning programs use video broadcast formats, in 
which teachers view lectures, demonstration classes, 
and other materials online. This approach allows an in- 
structor’s presentation to be broadcast to multiple sites, 
and, depending upon the available technology, for in- 
teractions via video conferencing, audio conferencing, 
or online text messaging. The disadvantage of this ap- 
proach is that it generally requires participants to go to 
a specific site in which the technology is available, so it 
involves scheduled sessions. However, as the technol- 
ogy and available bandwidth continues to advance, 
video conference-based approaches will become more 
widely accessible to teachers in their schools and 
homes. 

Another common e-learning provides individual- 

ized, self-paced instruction, in which each teacher pro- 
ceeds through a series of online learning activities at his 
or her own pace. For some limited topics, this takes the 
form of self-study without any interactions, but more 
often this approach involves some interactions with an 
instructor through an online discussion board, email or, 
in some cases, telephone. While this approach provides 
the most flexibility, it lacks opportunity for interactions 
with colleagues. 

Other approaches are emerging. For example, some e- 
learning programs are beginning to explore the use of 
voice-over-internet to enable participants to engage in 
spoken exchanges. There are also hybrid models that 
integrate onsite meetings, classroom visits, or local 
study groups into a primarily e-learning course. 

In addition to providing full courses, e-learning can be 
used to enhance and extend face-to-face workshops and 
courses, coaching and mentoring programs, teacher 
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study groups, and other professional development ap- 
proaches. In each case, the e-learning technology pro- 
vides a convenient means of communicating and 
sharing information, one that doesn’t depend on people 
being available at the same time or place. As an en- 
hancement to other types of professional development 
programs, e-learning can enable participants to con- 
tinue discussions from onsite meetings, provide access 
to experts and resources that are not available locally, 
enable rapid responses to questions, facilitate develop- 
ing collections of shared resources, and, in general, 
deepen connections with colleagues and mentors. 

Examples of e-Learning Programs for Educators 

e-Learning for educators has rapidly come into wide- 
spread use in the past few years (Galley, 2002). Accord- 
ing to the Association for Supervision and Curriculum 
Development, online courses are the fastest growing 
form of teacher training (Seal, 2003). A wide variety of 
innovative programs are available from states, districts, 
universities, colleges of education, regional service pro- 
viders, and for-profit and non-profit companies. As ex- 
amples, a few of the many innovative programs are 
briefly described below. 

Louisiana State Department of Education 
(www.doe.state.la.us/lde/) 

The Louisiana State Department of Education (LA- 
DoE) has been one of the early innovators in the use of 
e-learning in professional development. Some examples 
of e-learning in Louisiana statewide programs include 
the following: 

1. Bridging the Gap through Universal Design for Learn- 
ing (www.doe.state.la.us/lde/lcet/399.html) is a LA- 
DoE initiative for districts interested in addressing 
the challenges of the Individuals with Disabilities 
Education Act (IDEA) and learning how to make the 
general curriculum accessible for all learners. The 
program is designed for district or school teams 
comprised of special education teachers, regular edu- 
cation teachers, curriculum and special education su- 
pervisors, library media or technology specialists, 
and building or district administrators. It is offered 
in both face-to-face and in anytime, anywhere e- 
learning, with the same content presented in each. 
Since the program began in 2002, more than 200 K- 
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12 educators and university faculty have completed 
the online course 

2. LA-DoE also offers Effective Instructional Technology 
(www.doe.state.la.us/lde/lcet/1 82 1 .html) online 
courses to enable educators from across the state to 
meet national and state standards relative to technol- 
ogy. The first twelve-week online course helps 
teachers understand and apply the benefits of in- 
structional technology in their classrooms. The sec- 
ond course guides the teacher through the process of 
creating a portfolio that demonstrates their progress 
toward enhancing education through technology. 
These courses enable teachers to complete activities 
that address all of the International Society for 
Technology in Education (ISTE) National Educa- 
tional Technology Standards (NETS) for students 
and teachers. These courses also count toward re- 
licensure of a teaching certificate, earn university 
credit or Continuing Learning Units, and provide six 
of the nine hours needed to meet the requirements 
for certification in Educational Technology Eacilita- 
tion. Since this program begin in Eall 2003, about 20 
educational leaders have been trained as instructors 
in the program and more than 7 5 teachers have 
completed the first course. 

3 . The Louisiana Teacher Assistance and Assessment Program 
(LaTAAP) (www.doe.state.la.us/lde/pd/623.html) is a 
state-mandated induction program designed to sup- 
port new teachers in meeting the performance stan- 
dards required for certification. Each mentor is an 
experienced teacher who acts as a coach, models ef- 
fective practices, and helps the novice teacher formu- 
late a formal Professional Development Plan. With 
funding from NCLB Title IID grants, a number of 
Louisiana districts have added an e-learning compo- 
nent LaTAAP to create the EIRSTTech (Erame- 
work for Inducting, Retaining and Supporting 
Teachers with Technology) program. In these dis- 
tricts, the teachers and mentors are each given a lap- 
top computer and access to an e-learning environment 
that provides both on-demand resources for the new 
teachers and a communication channel that extends 
the mentoring opportunities through electronic ex- 
changes. Twenty-five of Louisiana’s sixty-eight school 
districts have participated in the EIRSTTech program 
since its inception in 2002. 
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4. The Louisiana Principal Induction Program (LPI) 
(www.doe.state.la.us/lde/lcet/1642.html), mandated 
for new principals and assistant principals, is de- 
signed to build the capacity of new building-level 
administrators to provide leadership to their schools 
in both instructional and administrative areas. In ad- 
dition to local, regional and state meetings, a major 
component of this two-year program consists of 
online modules based on the Standards for School 
Principals in Louisiana and other national leadership 
standards. These modules focus on teaching and 
learning, promoting school improvement, and en- 
hancing student achievement. The online activities 
are facilitated by team mentors and LA-DoE staff, 
and incorporate statewide discussions with inductees. 
During the past three years, more than 1,280 build- 
ing level administrators have completed the online 
component of the LPI program. 

5 . Experienced principals and district administrators 
are also provided with online learning opportunities 
through the state’s LEADTech (Louisiana Educational 
Advancement and Development with Technology) 
program (www.louisianaschools.net/leadtech/). LEAD- 
Tech, originally funded through the Gates 
Eoundation, provides administrators with an eight- 
week, graduate credit, online course designed to 
build strong instructional leaders who effectively use 
technology. Since the program began in spring 2000, 
more than 1,200 educational leaders and policymak- 
ers have successfully completed the online course. 

Florida Online Reading Professional Development Pro- 
gram (www.itrc.ucf.edu/forpd/about/) 

The Elorida Online Reading Professional Development 
program (EOR-PD) provides an online staff develop- 
ment course designed to serve as a primary delivery 
mechanism for improving teaching methods in preK- 
12 reading instruction. Developed by the University of 
Central Elorida with funding from the Elorida DoE, 
the goals of EOR-PD include the following: 

• support the Elorida Department of Education in its 
statewide implementation of a reading professional 
development system using online delivery; 

• serve as a model for reading professional develop- 
ment online delivery; 
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• translate scientifically-based research into action; 

• empower teachers to use innovative, creative, and 
effective strategies to help all children learn to read 
proficiently; 

• increase the teachers' knowledge base about reading; 
and 

• improve curriculum and reading instruction in order 
to improve student learning. 

The EOR-PD 14-week online course is facilitated by 
reading specialists and other qualified educators who 
receive training to become certified to teach the course. 
Since EOR-PD was first offered in January 2003, more 
than 6700 Elorida teachers have enrolled in the course. 
The course is offered for free to Elorida teachers. By 
arrangement with Elorida universities, teachers can ob- 
tain graduate credit, with a tuition fee required. 

An outside interim report of the first year of the EOR- 
PD course found that: 

• Over 90% of participants indicated that the value of 
reading strategies introduced in EOR-PD was excel- 
lent or good; 

• Approximately 97% of participants indicated EOR- 
PD was excellent or good in covering the state and 
national reading initiatives; 

• Over 90% of participants indicated that EOR-PD 
has contributed to their knowledge of effective read- 
ing theory, research, and instructional practice to an 
excellent or good extent; and 

• Over 87% of EOR-PD participants indicated they 
would make changes to their classroom reading in- 
struction as a result of EOR-PD. 

Milwaukee Public School District 

(www.milwaukee.k12.wi.us/pages/MPS/Teachers_Staff/ 

Tech_Tools/Portal) 

The Milwaukee Public School District (MPS) has es- 
tablished a strategic initiative, the Professional Support 
Portal, designed to use technology to support teacher 
induction, retention, and continued professional 
growth at all stages of the teaching career continuum. 
The goal of the Portal project is to build social and 
technical infrastructures that will help address three 
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categories of teachers’ needs: (1) access to classroom 
resources, lesson planning tools, and teaching and 
classroom management strategies; (2) opportunities to 
work with mentors and observe classrooms led by ex- 
perienced teachers; and (3) ongoing social, emotional, 
and professional support from peers and experienced 
teachers. The Portal project has allowed for a conver- 
gence of several key technology initiatives already un- 
derway, and has supported the initiation of new e- 
learning opportunities for both teachers and principals. 
Some of these include: 

• Guidance via videoconferencing from Harvard fac- 
ulty for a cohort of MPS principals who completed 
the Harvard Principal's Institute in Cambridge, MA 
this past summer and now are receiving feedback and 
support as they implement leading-edge practices in 
local settings; 

• The use of Tappedin®, a non-profit multi-user vir- 
tual environment for professional development de- 
veloped by SRI International (www.ti2.org). By 
providing an online conference center that combines 
tools for exchanging resources and for asynchronous 
and synchronous communications, Tappedin® en- 
ables Milwaukee educators to build and sustain 
communities of practice; 

• A series of online professional development work- 
shops. With training and support from the EDC 
EdTech Leaders® Online program, MPS staff has 
developed capacity to design, implement, and sustain 
a robust district-wide online professional develop- 
ment program (www.milwaukee.kl2.wi.us/pages/ 
MPS/T eachers_Staff/Training/ Continuing_ 
Education/Technology). Through this program, 
MPS has offered more than 50 workshops, with 
more than 1300 participants. These six-week work- 
shops, facilitated by MPS staff, have focused on the 
use of technology in different subject areas and 
grade-levels, and on the use of data-informed deci- 
sion making for teachers and administrators. 
Through an arrangement with Cardinal Stritch Uni- 
versity, a local Institution of Higher Education with 
close teacher preparation and professional develop- 
ment ties to the district, participants in these work- 
shops can receive graduate credit. 
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Lesley University (www.lesley.edu/online_courses.html) 

Lesley University (previously Lesley College), based in 
Cambridge Massachusetts, is the largest teacher certifi- 
cation program in Massachusetts and the largest gradu- 
ate program for educators in the United States. In 
addition to traditional, on-campus programs, Lesley 
University offers Bachelor’s and Master’s degree pro- 
grams in education designed to provide flexibility to 
meet the needs of students who have responsibilities 
that make it impossible for them to attend regular on- 
campus courses. These programs, which have a 20-year 
history at Lesley University, are currently provided at 
learning sites in 2 1 states. They use a weekend study 
format in which 45 hours of class time are provided in 
two intensive weekend sessions, with class assignments 
completed before, between, and after these sessions. 
Students in these degree programs are grouped into 
cohorts that progress through the course sequence to- 
gether so that each cohort forms an ongoing learning 
community. Lesley University also provides online li- 
brary resources and other supports to make the off-site 
weekend program experience as similar as possible to 
the experience of its on-campus students. 

The Lesley University off-site model provided a natu- 
ral basis for offering e-learning programs and Lesley 
University has, since 1997, offered an online Master’s 
program in Technology in Education. Courses are 
spread over 12 weeks, rather than packed into two in- 
tensive weekends, with weekly sessions involving read- 
ings, assignments, and online, asynchronous 
discussions. Students can proceed through the program 
at their own pace but find that an online learning 
community develops within each course. Students re- 
ceive feedback from faculty on their work, and faculty 
are available for online and phone exchanges outside of 
class. A recent survey found that students in the online 
program rated their experience to be very positive. 
They rated the frequent interactions with the instruc- 
tors as most valuable, followed by the up-to-date and 
relevant content and the interactions with their fellow 
students. 

Lesley University also offers a new online Master’s degree 
in K-8 science education program, developed in collabo- 
ration with TERC (www.lesley.edu/soe/science/ 
ts_indepth.html). Program participants build their un- 
derstanding of core science concepts while engaging in 
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the same learning paradigm of inquiry-based science 
that they will bring to their classrooms. Approximately 
50% of each course in this program is devoted to learn- 
ing science content by doing science, 25% to consider- 
ing issues of pedagogy, curriculum, and assessment, and 
2 5 % to trying ideas in the classroom and reflecting on 
these experiences with other program participants. 

Each module is taught by two instructors: a scientist 
well versed in the science domain and a science educa- 
tor who supports participants as they consider peda- 
gogical strategies for bringing science inquiry into their 
classrooms. Participants use threaded discussions, an 
electronic meeting place, images, videos, text, and cur- 
rent online data and information. Instructors track 
learners' progress through an online archive of partici- 
pant portfolios. Research comparing this program to 
the Lesley University on-campus equivalent program 
showed better learning results for the online students in 
understanding the inquiry process and classroom use of 
inquiry methods. The research also showed that the 
online students and instructors both spent more time 
on the course that the on-campus group (Harlen & Al- 
tobello 2003). 

Western Governors University Teachers College 
(www.wgu.edu/education/landing.asp) 

WGU is a true virtual university: while its administra- 
tive officers are based in Salt Lake City, it has no cam- 
pus and its faculty are distributed around the country. 
Lounded by the governors of 19 western states as a 
non-profit university, WGU is the only accredited uni- 
versity in the U.S. offering competency-based, online 
degrees. The WGU Teachers College offers Bachelor’s 
degree teacher preparation programs and post- 
baccalaureate licensure programs to prepare students 
for licensure as preK-8 teachers, or as middle school or 
high school mathematics, social studies, or science 
teachers. It also offers Master’s degree programs in the 
same areas of specialization. 

Each student is guided by a WGU mentor. Lollowing a 
pre-assessment and interview, the student and mentor 
develop an individualized academic action plan that 
guides the student in selecting online courses, inde- 
pendent study modules, or other educational experi- 
ences to prepare for the competency-based assessments. 
The courses and other learning activities in the aca- 
demic action plan are chosen from a wide variety of 
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WGU-approved online courses available from colleges, 
universities, and other providers. That is, WGU does 
not have its own catalog of courses; the course options 
for its students are distributed among many course pro- 
viders. WGU Teachers Colleges also arranges for stu- 
dents to do trial teaching and demonstration teaching 
(equivalent to student teaching) in a school district in 
the student’s area, and arranges for a field supervisor to 
monitor and assess the student’s classroom practice. 

All WGU programs are competency-based, rather than 
course or credit based. The WGU Teachers College 
has developed a comprehensive set of competency 
standards for teachers that includes those found in 
many state and national standards. Based upon these 
standards, they developed a comprehensive assessment 
system to evaluate each student’s subject area knowl- 
edge, conceptual foundations of teaching, and teaching 
practices. The assessments involve a combination of 
proctored examinations to assess knowledge, perform- 
ance tasks to assess skills, and observations of classroom 
practices to assess teaching ability. A student who suc- 
cessfully completes all the required assessments is eligi- 
ble for his or her degree, no matter how the student has 
acquired the knowledge and skills assessed. Therefore, 
students can take advantages of their existing competen- 
cies and advance through their program at their own rate 

As part of their program, candidates enrolled in a 
WGU teacher preparation program plan and teach a 
four-week standards-based instructional unit that pro- 
vides an opportunity to demonstrate that they can inte- 
grate all elements of teaching and positively influence 
student learning. They complete a Teacher Work Sample 
professional portfolio that provides direct evidence of 
the candidate's ability to design and implement a multi- 
week standards-based unit of instruction, assess student 
learning, affect student achievement and then reflect on 
the learning process. 

The WGU programs are designed for highly motivated 
adults who want to become certified teachers, or for 
certified teachers who want to complete their Master’s 
degree. Success in these programs requires an ability to 
work independently and a comfort with online com- 
munication. It does not replace a typical college experi- 
ence, but provides a new, flexible option for many 
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adults who need to fit continuing education around 
work and family commitments. 

The infrastructure of the WGU Teachers College has 
been designed to allow for rapid growth to a large 
number of students. As of June 2004, WGU Teachers 
College had approximately 1600 students enrolled in its 
degree programs. Students can start at the beginning of 
any month, and currently 200 to 300 students are being 
added each month. 

Other e-Learning and Teacher Quality Programs 

There are many additional program throughout the 
country involving e-learning and teacher quality. A few 
examples include: 

The Southern Regional Education Board (SREB) Multi- 
State Online Professional Development Consortium 
(www.sreb.org/programs/EdTech/toolkit/onlineindex.asp) 

The Southern Regional Education Board (SREB), 
working in collaboration with the 16 states in its re- 
gion, the SouthEast Initiatives Regional Technology in 
Education Consortium (SEIRTEC), and EDC, has es- 
tablished the Multi-State Online Professional Devel- 
opment Consortium to promote and support efforts of 
states and schools to use online professional develop- 
ment. All 16 SREB states' departments of education are 
working together to use and promote the use of the 
Web to provide needed quality professional develop- 
ment for teachers regardless of where they live or work. 

Virginia Regional Consortia 

In Virginia, several regional consortia are building e- 
learning collaborations, with the support of Title IID 
fands, to share online professional development re- 
sources and workshops. The Blue Ridge West Consor- 
tium (www.scsb.org/etlo_workshops.htm), encompassing 
19 school divisions, will offer approximately 75 online 
workshops for teachers during the next year in a wide 
range of subject areas and grade levels. A similar pro- 
gram is available through the Virginia North Tier 
Partnership (www.culpeperschools.org/ETLO), which 
will offer 50 online workshops to teachers across their 
1 3 school district consortium. In both consortia, the 
workshops are a component of their teacher quality ini- 
tiatives and college credit is available for participants. 
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Michigan LearnPort (http://www.learnport.org/) 

The Michigan Department of Education and the Michi- 
gan Virtual University, with support from the legislature 
and governor, is creating the Michigan LearnPort, a state- 
wide portal designed to help both teachers and education 
para-professionals meet the Michigan quality standards. 
LearnPort is designed to effectively disseminate informa- 
tion to all educators in the state, provide a repository of 
educational resources to which teachers can contribute, 
and provide a central resource for online professional de- 
velopment. LearnPort will offer all educators in Michigan 
free, five hours of online learning that will introduce them 
to e-learning and enable them to become better consum- 
ers of online education. It is also working with universi- 
ties, colleges, and school districts within Michigan to 
develop a catalog of e-learning courses for teachers and 
para-professionals. 

Los Angeles Unified School District 
(www.lausd.k12.ca.us/lausd/offices/opd/) 

The Los Angeles Unified School District (LAUSD) has 
incorporated e-learning into its ongoing professional 
development and training programs. Since October 
2000, more than 25 online professional development 
specialists have been trained to design and deliver 
online workshops that are aligned with district stan- 
dards and address a range of subject areas and grade 
levels. Each of these workshops includes six weeks of 
online instruction and two face-to-face meetings. In the 
fall of 2003, LAUSD expanded use of online learning 
to students, and launched the Los Angeles Virtual 
Academy (LAVA). 

PBS TeacherLine (www.pbs.org/teacherline) 

Eunded by the U.S. Department of Education, PBS 
TeacherLine has worked with a variety of content de- 
velopers to create a catalog of online professional de- 
velopment courses for teachers, focusing on 
mathematics education, teaching reading, and effective 
uses of technology in the classroom. These courses are 
made available through local PBS affiliate stations and 
through other partnerships. 

Lessons Learned about e-Learning for 
Educators 

Most research studies on e-learning are based on 
higher education courses, with impact on learning 
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measured by tests and grades (Phillips and Merisotis, 
1999). The most common result of this research is to 
find no significant differences between student learning 
in face-to-face and online courses, and to find that 
those who take online versions of courses are as satis- 
fied with the experience as those who attended classes 
(Russell, 1999). However, in some studies, such as 
Koory’s (2003) comparison of an Introduction to 
Shakespeare course delivered online and face-to-face, 
and Harlen & Altobello’s (2003) comparison of an 
online and face-to-face science education course, the 
results showed better learning outcomes online. 

Survey data from participants in many online profes- 
sional development programs show that when online 
courses are well-designed and implemented, partici- 
pants report them to be valuable and enjoyable learning 
experiences that impact both knowledge and profes- 
sional practice. That is, the participant survey results 
from many programs are consistent with those from the 
Florida and Lesley University programs summarized 
above. Experimental comparisons of different models 
of e-learning and onsite professional development for 
educators are underway, but results are not yet avail- 
able. 

The available data, along with analyses of the e- 
learning programs on which they are based, lead to the 
conclusion that well-designed e-learning programs can 
have positive impact on educators’ knowledge and prac- 
tices, and the amount of impact is comparable to that in 
other professional development approaches. It appears 
that the quality of the course content and design, and 
the nature of the interactions with the instructor, are 
more important determinants of learning than whether 
the course is taught face-to-face, online, or some blend 
of both (Koory 2003). If this is verified by future re- 
search, e-learning will become an increasingly impor- 
tant tool to help meet teacher quality requirements, 
one that can provide professional development oppor- 
tunities not otherwise available and that can also be 
used to enrich other forms of professional development 
by adding online resources and communications. 

However, e-learning is not a magic solution, and many 
educators have already experienced poor quality e- 
learning offerings. Some of the lessons learned about 
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providing effective e-learning for educators include the 

following: 

• Successful e-learning programs need to be designed 
to incorporate principles of effective professional de- 
velopment and to take good advantage of the oppor- 
tunities afforded by the technology, such as anytime, 
anyplace access to discussions, resources and experts. 

• The e-learning format must match the goals of the 
course and the needs of the participants. For exam- 
ple, specific skills and knowledge can be learned by 
online^ self-paced courses^ but changing classroom prac- 
tices generally requires an online learning community 
approach in which teachers view model practices (in 
person or via video), experience using them in the 
classroom, reflect upon their experience, and engage 
in discussions with peers and mentors. 

• The nature of the online interactions between learn- 
ers and instructors, and among the group of learners, 
is critical. Courses that are highly rated by partici- 
pants generally involve many, content-rich interac- 
tions with the instructor and with other participants. 

• Developing effective, multi-media, online learning 
experiences for educators is both challenging and ex- 
pensive. It requires knowledge of the subject matter, 
adult learning theory, course design, multi-media 
technology development, and the specifics of online 
learning. Quick conversions of successful face-to- 
face courses and workshops to the online medium 
general result in poor quality e-learning. 

• Teaching online is different from teaching face-to- 
face, and instructors who teach online should receive 
training in online communications and course facili- 
tation. 

• e-Learning has a different cost structure than onsite 
courses, is not necessarily less expensive. While there 
are savings in physical space and perhaps travel, 
there are costs for the technology infrastructure and 
course design. Many instructors find that teaching 
online takes more time than teaching a comparable 
class face-to-face, given the nature and amount of 
online exchanges that occur. Therefore, online 
instructors, in interactive formats, cannot handle 
more students than in a comparable onsite class. 
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• Participating teachers need support and incentives, 
just as they do for other types of professional devel- 
opment. Districts that view e-learning as a low-cost 
alternative because teachers can do it on their own 
time, allowing the district to avoid providing release 
time and pay for substitutes, generally find resistance 
from their teachers and the teachers’ union. 

• Ready access to reliable technology with the needed 
capabilities — and to technical support when 
needed — is essential for participants to have good 
experiences with e-learning. It is essential that the 
technology demands of the e-learning program do 
not exceed the technology available to participants at 
both school and home. 

• There are a number of e-learning technology sys- 
tems available, with Blackboard, WebCT, eCollege, 
Desire2 Learn and ANGEL among those that are 
commonly used in programs for educators. While 
each system has some strengths and weaknesses 
compared to the others (and each keeps releasing 
new versions with additional features), the quality of 
course design and of online teaching is far more im- 
portant than which e-learning platform is used. 

• When possible, hybrid programs that blend e- 
learning with face-to-face meetings, study groups, 
coaching, and other activities result in the most ef- 
fective professional development programs. 

• e-Learning may not be for everyone. Some educators 
report that they are simply uncomfortable communi- 
cating online or unable to organize their time and 
work without face-to-face meetings. 

• e-Learning provides teachers with a chance to ex- 
perience for themselves new ways of learning, which 
can inform their decisions about the use of technol- 
ogy with students. 

• e-Learning provides more opportunities for account- 
ability and quality control than other forms of pro- 
fessional development. By its very nature, e-learning 
leaves a detailed record of content, assignments, 
products, and interactions that can be reviewed to in- 
sure quality. A discussion in an onsite classroom van- 
ishes as it is produced; an online discussion in an e- 
learning course is preserved for as long as desired. 
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and the contribution of each individual can be as- 
sessed. e-Learning systems thereby capture a great 
deal of information that can be used to assess partici- 
pation and learning, as well as the quality of teach- 
ing. 

Policy Issues 

e-Learning also brings challenges to policymakers. 
These challenges can be summarized by the need for 
policies that support the development of e-learning re- 
sources and programs, remove barriers to e-learning, 
provide funding for innovation and research, and en- 
sure high-quality e-learning programs for educators. 

The NCLB teacher quality requirements place a strong 
emphasis on subject area content knowledge. Programs 
that provide online content in courses designed for 
teachers, such as the Lesley University K-8 science 
education program, have shown good initial success. 

But many more high-quality, content-rich, educator- 
oriented e-learning courses are needed to meet the pro- 
fessional development demands driven by NCLB. 
There are many potential providers of these courses: 
universities, colleges of education, state departments of 
education, school districts, regional service providers, 
online learning consortia, national grant-funded pro- 
grams (such as PBS TeacherLine), education publish- 
ers, and other private sector organizations, will all be 
needed to help meet this demand in a timely way. De- 
veloping high quality online professional development 
programs is expensive, and support for this work is es- 
sential to the success of using e-learning to help address 
the teacher quality challenges on a national scale. 

e-Learning providers are finding a number of policy 
barriers. For example, e-learning is very compatible 
with competency-based or knowledge-based assess- 
ments for teachers, but not with traditional measures 
such as number of contact hours. It is also not compati- 
ble with accreditation criteria that require buildings, 
physical libraries, and on-site faculty. e-Learning pro- 
grams can easily cross state boundaries, but state certi- 
fication systems don’t always have a place for courses 
and programs offered from other states. Policies regu- 
lating incentives for teachers are also critical, as teach- 
ers engaged in e-learning professional development 
need to have access to the same stipends, credits, re- 
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lease time, salary increases, and other incentives as 
teachers engaged in more traditional forms. e-Learning 
for teachers also raises equity issues: Will all teachers 
have access to the needed technology to participate in 
this form of professional development? 

We are just beginning to explore the potential of e- 
learning and learning to understand how to use it most 
effectively. In addition, as more advanced technologies 
become widely available to educators, the opportunities 
for e-learning and the need to explore potential uses of 
these technologies will increase. Some examples in- 
clude real-time virtual visits to classrooms, use of video 
that can be annotated by course participants, voice- 
based online discussions, digital portfolios, and more 
sophisticated online collaborative work tools. Support 
is needed to develop innovative models of e-learning 
for educators and to conduct research to test the effec- 
tiveness of these models. 

Quality control policies and procedures are critical for 
the widespread success of e-learning. For some, the 
reputation of e-learning has already been damaged by 
poor quality courses that have been marketed to educa- 
tors. In some places, the response to experiences with 
poor quality e-learning has been to not accept e- 
learning as a valid form of professional development, 
rather than to put the needed quality assurance proc- 
esses in place. In addition, since e-learning is new, most 
educators and policymakers are not yet prepared to be 
intelligent consumers of the many formats of e- 
learning, and this problem will increase as more sophis- 
ticated technologies are employed. In addition, many 
have underestimated the expertise, time, and costs in- 
volved in developing high quality e-learning courses 
and programs. While several groups have developed e- 
learning guidelines (e.g., ITRC 2000, NSDC 2000, 
NASBE 2001), these have not yet been incorporated into 
widely-used quality assurance policies and procedures. 

Conclusions 

The challenges of meeting the teacher quality require- 
ments are huge, and the time to meet them is short, e- 
Learning can clearly provide a valuable means to help 
us meet these challenges, and the success of well- 
designed and well-implemented e-learning for educa- 
tors has been demonstrated. e-Learning is ideal for in- 
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creasing professional development opportunities for 
current teachers and for strengthening the preparation 
of future teachers. The anytime, anyplace nature of e- 
learning provides flexibility that can expand access to 
those in rural areas and to those whose schedules are 
filled with other responsibilities. In addition, e-learning 
can be blended with other professional development 
approaches to extend opportunities for educators to 
communicate and share resources. Much still needs to 
be done, however, before all educators will have suffi- 
cient access to high-quality e-learning courses that ad- 
dress the specific content and competencies they need 
to meet their state teacher quality standards. To meet 
this goal, policy makers need to support the develop- 
ment of e-learning resources and programs; remove 
barriers to e-learning; provide funding for innovation 
and research; and ensure high-quality e-learning pro- 
grams for educators 
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“ NCLB includes the following as core academic sub- 
jects: English, reading or language arts, math, science, 
history, civics and government, geography, economics, 
the arts, and foreign language. Special education 
teachers and teachers of English language learners must 
be highly qualified if they teach core academic subjects 
to their students. 



14 



U.S. Department of Education Secretary’s No Child Left Behind Leadership Summit 

Increasing Options Through e-Learning 

86 



